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j  The  objsctive  of  this  -research  uias  tD  determine  what 
information  the  current  Air  Force  contract  programmers  in 
the  field  perceive  should  be  included  in  a  handbook  designed 
to  help  new  personnel  learn  the  job  of  contract  programming 
as  quickly  as  possible.  A  census  utas  conducted  using  the 
mail  survey  technique.  Responses  to  the  survey  uBre 
tabulated  and  analyzed  using  various  statistical  techniques. 
This  research  resulted  in  the  identification  of  15 
information  topic  areas  which  could  be  incorporated  into  a 
contract  programmer's  handbook.  In  addition,  the  research 
pointed  out  the  need  for  such  a  handbook  as  indicated  by  the 
high  return  rate  of  the  survey .  / 
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V 


INFORMATION  REQUIREMENTS  FOR 


THE  NEW  CONTRACT  PROGRAMMER 


Overview 

Base  level  civil  engineering  contract  programming  is 
vital  to  the  mission  of  Air  Force  bases  located  throughout 
the  world.  Contract  programming  personnel  program 
maintenance,  repair,  and  construction  projects  for  the  bases 
so  that  present  and  future  mission  requirements  can  be  met. 

Contract  programmers  collect  information  and  prepare 
documentation  to  support  such  vital  programs  as  the  Military 
Construction  Program,  the  Operations  and  Maintenance 
Program,  the  Nonappropriated  Funds  Program,  and  the  Military 
Family  Housing  Program  for  the  various  bases.  These 
programs,  along  with  others,  result  in  the  maintenance, 
repair,  and  construction  of  facilities  and  utilities  located 
throughout  the  Air  Force. 

The  Job  of  contract  programming  is  very  dynamic,  and 
requires  the  programmer  to  be  flexible  enough  to  adjust  to 
frequent  changes  in  programming  requirements  and  guidance. 
These  changes  can  be  brought  about  by  many  factors, 
including  changes  in  Major  Command  (MAJCOM)  and  Headquarters 
United  States  Air  Force  (HQ  USAF)  programming  guidance. 
Whatever  the  reason,  contract  programmers  must  know  their 


job  well  enough  to  adjust,  oftentimes  quickly,  to 
programming  changes  and  maintain  program  continuity. 

Personnel  newly  assigned  to  contract  programming  should 
learn  the  job  of  contract  programming  as  quickly  as 
possible.  An  effective,  productive  programmer  is  an 
individual  who  can  produce  quality  project  documentation 
while  responding  to  program  requirements  that  are 
continually  changing.  A  programmer  must  be  effective  so 
that  vital  projects  are  not  delayed  due  to  the  programmer's 
inability  to  adjust  to  changing  requirements.  A  further 
discussion  of  the  job  of  contract  programming  as  well  as  the 
roles  and  responsibilities  of  the  base  IbvbI  civil 
engineering  squadron  is  presented  in  Chapter  II. 


Often  personnel  newly  assigned  to  programming  are  left 
to  "learn  by  doing"  the  Job  of  contract  programming.  They 
stumblB  through  by  the  process  oF  trial  and  error.  They  are 
told  to  read  regulations  pertaining  to  contract  programming 
and  ask  questions  of  othBr  contract  programmers.  This 
process  of  learning  can  be  very  slow  and  frustrating  for  the 
new  contract  programmer . 

Even  though  trial  and  error  and  "learn  by  doing”  are 
perhaps  necessary  to  learn  the  Job  of  contract  programming, 
any  information  provided  to  the  newly  assigned  individual 
which  will  help  him  or  her  learn  the  job  Faster  will  be  of 
benefit  to  the  individual  as  well  as  to  the  contract 


programming  office  and  the  installation. 

Objective  of  the  Research 

The  objective  of  this  research  uas  to  determine  what 
general  background  information  the  current  Air  Force 
contract  programmers  in  the  field  perceive  should  be 
included  in  a  handbook  designed  to  help  new  personnel  lBarn 
the  job  of  contract  programming  as  quickly  as  possible.  The 
approach  to  this  study  uas  to  survey,  using  a  questionnaire, 
the  current  Air  Force  contract  programmers  in  the  field. 

This  research  uas  intended  to  bB  the  first  step  towards  the 
publication  of  a  handbook  for  new  contract  programmers. 

Scope  and  Limitations  of  Study 

The  scope  of  this  study  is  limited  to  the  following 
items: 

1.  The  perceptions  of  U.S.  Air  Force  base  level  and 
nAJCOn  level  civil  engineering  contract  programmers 
within  the  continental  United  States  CCDNUS) . 

2.  The  role  of  the  contract  programmer  in  a  peacetime 
environment . 

3.  The  collection  and  tabulation  of  information 
obtained  from  the  questionnaires  into  lists 
identifying  thB  information  needed  to  help  new 
contract  programmers  learn  the  job  as  quickly  as 
possible . 

4.  Preparation  of  a  list  of  information  topic  areas 
which  could  be  included  in  a  handbook. 


Assumptions 


This  research  effort  is  based  upon  thB  following  two 
assumptions: 

1.  Information  collected  from  the  current  experienced 
and  inexperienced  Air  Force  base  level  and  flAJCOFI 
level  contract  programmers  in  the  CONUS  is  adequate 
to  solve  the  research  problem. 

2.  The  information  collected  will  identify  what 
general  background  information  an  individual  who 
has  no  prior  experience  at  contract  programming 
needs  to  learn  the  job  as  quickly  as  possible. 


1 1 .  Background 


IntLBdycliflD 

The  U.S.  Air  Force  is  charged  with  the  responsibility 

for  accomplishing  the  following: 

The  Air  Force  prepares  aerospace  forces  to 
perform  offensive  and  defensive  operations  with 
the  purpose  of  defending  the  United  States, 
deterring  aggression,  and  bBing  ready  to  conduct 
warfare  to  support  national  objectives  Cl: 3-1). 

It  executes  this  tasking  through  the  accomplishment  of 

various  missions.  Each  of  the  U.S.  Air  Force  bases  located 

throughout  the  world  is  responsible  for  accomplishing  one  or 

more  of  these  missions.  These  missions  may  involve  support 

for  weapon  systems  such  as  tactical  fighters,  strategic 

missiles,  or  long  range  bombers  or  support  for  personnel 

training,  radar  surveillance,  and  other  areas. 

This  chapter  begins  with  a  presentation  of  the 

organizational  structure  of  a  typical  Air  Force  base  that  is 

responsible  for  accomplishing  a  flying  mission.  Then  a 

brief  presentation  of  the  organizational  structure,  roles, 

and  responsibilities  of  the  base  civil  engineering 

organization  is  presented.  This  is  followed  by  a  discussion 

of  the  job  of  contract  programming.  The  goal  of  this 

chapter  is  to  help  an  individual  who  is  unfamiliar  with  Air 

Force  civil  engineering  to  better  understand  the  role  that 

civil  engineering,  and  ultimately  the  contract  programmer, 

plays  in  supporting  the  Air  Force  mission(s)  of  the  base. 


Qvscyiati. 

The  major  organizational  element  for  a  typical  Air 
Force  base  with  a  flying  mission  is  thB  ’’Wing”.  The  Wing 
Commander  is  the  senior  individual  responsible  for  ensuring 
that  the  assigned  aircraft  Ce.g.,  tactical  fighters,  long 
range  bombers,  etc.)  are  mission  capable;  that  is,  ready  to 
launch  and  capable  of  accomplishing  the  mission.  He 
accomplishes  this  with  the  support  of  personnel  who  work 
directly  for  him  as  well  as  with  thB  support  of  personnel  in 
other  organizations  on  the  base. 

The  wing  is  comprised  of  subordinate  units  which  are 
organized  under  four  main  functional  areas:  aircraft 
operations,  aircraft  maintenance,  resource  management,  and 
combat  support .  The  Combat  Support  Group  Commander  is 
responsible  for  providing  essential  base  support  services. 
Security  police,  base  civil  engineering,  and  the  Judge 
Aavocate  General  are  among  the  organizations  which  comprise 
the  Combat  Support  Group. 

BasB  Civil  Engineering 

The  primary  mission  of  the  base  civil  engineering 
organization  is  to 

provide  the  necessary  assets  and  skilled 
personnel  to  prepare  and  sustain  global 
Installations  as  stationary  platforms  for  the 
projection  of  aerospace  power  in  peace  and  war 
C2:B) . 

The  base  civil  engineer  (BCE)  is  responsible  for 
maintaining,  repairing,  and  replacing  facilities  and 


utilities  (water,  gas,  electricity,  etc.')  located  on  the 
base.  In  addition,  the  BCE  provides  fire  protection, 
manages  and  maintains  military  family  housing  on  the  base 
and  provides  other  services  as  well  (13:1-2). 

The  BCE  organization  is  made  up  of  eight  branches: 
Readiness  Management,  Industrial  Engineering,  Family  Housing 
Management,  Squadron  Section  and  Administration,  Financial 
Management,  Fire  Protection,  Operations,  and  Engineering  and 
Environmental  Planning  (6:3). 

The  Readiness  Management  branch  is  responsible  for 
ensuring  that  civil  engineering  personnel  are  properly 
trained  and  equipped  to  respond  tD  natural  disasters,  major 
accidents  and  most  importantly,  contingency  situations 
( combat ) . 

The  Industrial  Engineering  branch  "is  responsible  for 
evaluating  the  quality  of  work  performed  by  all  civil 
engineering  personnel”  (13:5).  The  industrial  engineer 
looks  for  mare  effective  and  efficient  ways  for  civil 
engineering  personnel  to  do  their  jobs. 

The  Family  Housing  Management  branch  manages  on-base 
housing  maintenance  and  assignments  and  maintains  current 
information  on  off-base  housing  that  is  available  within  the 
local  area  of  the  base. 

The  Squadron  Section  and  Administration  branch  handles 


personnel  matters  within  the  civil  engineering  organization. 
They  control  the  administrative  work  involved  with 
disciplinary  actions,  awards,  evaluation  reports,  and  other 


matters  pertaining  to  personnel  within  the  organization. 


The  Financial  Management  branch  maintains  control  over 
the  receipt  and  expenditure  of  funds  within  the 
organization . 

The  Fire  Protection  branch  functions  as  the  base  fire 
department.  Fire  detection,  suppression  and  crash  rescue 
capabilities  are  provided  by  this  branch.  In  addition,  fire 
prevention  training  is  also  conducted  for  all  base 
personnel . 

The  Operations  branch  of  civil  engineering  is  typically 
the  largest  branch  in  terms  of  personnel  assigned.  It  is 
responsible  for  the  day-to-day  maintenance  and  repair  of  the 
facilities  and  utilities  on  the  base.  Work  requirements 
which  exceed  the  capabilities  of  this  branch  in  terms  of 
scope,  cost  or  expertise  are  usually  accomplished  by 
contract  through  the  Engineering  and  Environmental  Planning 
branch . 

The  Engineering  and  Environmental  Planning  branch  is 
made  up  of  four  sections:  Engineering,  Contract  Management, 
Real  Estate  Management,  and  Environmental  and  Contract 
Planning . 

The  Engineering  section  is  responsible  for  the 
technical  design  of  maintenance,  repair,  construction  and 
service  contract  projects.  This  includes  preparing 
specifications,  drawings,  cost  estimates  and  statements  of 
work  for  the  projects. 

The  Contract  Management  section  is  responsible  for 
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ensuring  that  work  done  by  contractors  is  adequate  and 
acceptable  to  the  government. 

The  Real  Estate  Management  section  is  responsible  for 
maintaining  and  updating  real  estate  records  pertaining  to 
base  land  use  and  facilities. 

The  Environmental  and  Contract  Planning  section’s 
responsibilities  include  the  management  of  environmental 
resources  (land,  water  and  air  quality  and  hazardous  waste 
disposal) ,  the  preparation  and  implementation  of  the  base 
comprehensive  plan,  and  the  programming  of  maintenance, 
repair,  construction  and  service  contract  projects.  The 
contract  programmer  is  assigned  to  this  section. 

Contract  Programming 

Air  Force  Regulation  CAFR)  85-10,  Operation  and 

Maintenance  of  Real  Property,  defines  the  Job  of  contract 

programming  as  follows: 

Cl)  Establishes  and  maintains  facility 
requirements  needed  to  accomplish  the 
installation  mission. 

C2)  Develops  programming  documents  for  the 
construction,  maintenance,  and  repair  of  real 
property  facilities. 

(3)  Develops  and  performs  status  reporting  and 
control  of  related  projects  C2:23. 

Furthermore,  AFR  85-10,  Attachment  1  continues: 

Contract  planning  C programming!  prepares  and 
maintains  plans  and  programs  for  all  work  done  by 
contract  Ci.e.,  MFH,  NAF,  MCP,  Q&M,  etc.). 

Prepares  DD  Forms  1391/1391c  and  submits  to 
higher  headquarters  based  on  funding  needs. 


Identifies  to  the  funds  manager  budget 
information  for  contract  requirements  [2:143. 

This  regulation  also  refers  to  the  requirement 

"Prepares  and  maintains  thB  Maintenance,  Repair  and  Minor 

Construction  Program  Report  CMAREMIC)”  CE:14).  SincB  the 

regulation  uias  published,  MAREMIC  has  been  replaced  by  the 

Civil  Engineering  Contract  Reporting  System  CCECORS)  which 

is  prepared  and  maintained  by  the  contract  programmer . 

CECORS  is  a  computer  based  management  information  system 

which  contains  data  on  all  projects  identified  for  contract 

accomplishment  C present  and  future)  . 

As  stated  previously,  contract  programmers  collect 

information  and  prepare  documentation  to  support  such  vital 

programs  as  the  Military  Construction  Program  and  the 

Operations  and  Maintenance  Program  for  the  various  bases. 

These  programs,  along  with  others,  result  in  the 

maintenance,  repair,  and  construction  of  facilities  and 

utilities  located  throughout  the  Air  Force.  These  programs 

are  necessary  to  ensure  that  present  and  future  mission 

requirements  for  the  base  are  met. 

The  fallowing  is  an  example  of  how  a  contract 

programmer  might  program  a  project.  The  example  is  meant  to 

reflect  a  typical  situation  whereas  the  actual  programming 

process  may  vary  from  base  to  base . 

The  base  supply  officer  has  identified  a  need  for  an 

additional  warehouse  to  store  spare  parts  and  supplies  for 

the  tactical  aircraft  assigned  to  the  base.  He  submits  his 


request,  in  writing,  For  a  new  Facility  (including 
approximate  size  and  justiFication  For  the  warehouse)  to  the 
base  civil  engineering  squadron. 

The  request  is  received  and  evaluated  by  personnel  in 
the  Requirements  section  in  the  Operations  branch  oF  the  BCE 
organization.  It  is  determined  that  the  Operations  branch 
does  not  have  the  capability  (time,  manpower,  approval 
authority)  to  accomplish  the  request.  The  request  is  then 
Forwarded  to  the  contract  planning  section  For  contract 
accomplishment . 

The  contract  programmer  receives  the  request  and 
veriFies  that  the  size  oF  the  Facility  requested  Falls 
within  the  acceptable  Air  Force  guidelines.  He  then 
coordinates  with  the  Real  Estate  Management  section  to 
determine  whether  or  not  another  Facility  on  the  base,  not 
currently  in  use  or  only  partially  occupied,  can  be  used  as 
a  warehouse.  IF  no  suitable  Facility  can  be  Found,  he 
begins  programming  the  project. 

The  contract  programmer  classiFies  the  project  as  to 
the  type  oF  work  to  be  accomplished  and  determines  the 
programming  avenue  (Funds  source)  to  be  used.  In  this 
example,  the  project  would  be  classiFied  as  construction  and 
iF  it  is  large  enough  in  terms  oF  cost,  would  probably 
require  Military  Construction  Program  (MCP)  Funding. 

Next,  the  contract  programmer  prepares  a  preliminary 
cost  estimate  For  the  project.  He  then  prepares  the 
programming  documents  (BO  Form  1331/13910  which  include  the 


cost,  size,  description  of  construction,  and  justification 
for  the  project. 

The  project  is  then  placed  in  the  existing  HCP  project 
list  which  contains  other  projects  previously  identified  and 
programmed  for  the  base.  This  list  is  then  presented  to  the 
base  Facilities  Board  which  is  chaired  by  the  wing 
commander.  The  Facilities  Board  determines  the  relative 
priority  of  the  projects,  placing  those  projects  most 
critical  to  mission  accomplishment  higher  on  the  list. 
Seldom,  if  ever,  does  a  base  receive  sufficient  funds  to 
construct  all  projects  identified  for  a  given  fiscal  year; 
therefore  the  need  to  prioritize  the  projects. 

After  the  project  has  been  prioritized  by  the 
Facilities  Board,  the  contract  programmer  accomplishes 
additional  documentation,  such  as  environmental  and  economic 
analyses,  cost  estimate  refinements,  and  site  plans.  This 
documentation  is  then  submitted  to  higher  headquarters. 
Typically,  the  programmer  working  this  program  may  have  up 
to  HO  projects  at  any  one  time  in  a  five  year  program,  with 
each  fiscal  year’s  projects  undergoing  a  different  phase  of 
project  documentation. 

This  simple  example  of  project  programming  was 
presented  to  give  the  reader  an  idea  of  the  work  done  by  a 
contract  programmer  for  one  type  of  programming  avenue 
(f1CP)  .  The  actual  programming  process  is  much  more 
complicated  and  diverse,  requiring  various  types  of 
documentation  and  usually  requiring  other  sections  and 


IS 


branches  of  civil  engineering,  along  with  other  base 
organizations,  to  participate  in  the  process. 

Summary 

Contract  programmers  work  in  the  Environmental  and 
Contract  Planning  section  of  the  base  civil  engineering 
squadron.  Contract  programmers  are  tasked  with  collecting 
information  and  preparing  documentation  to  support  facility 
projects  which  are  vital  to  the  mission  of  the  base. 
Contract  programmers  are  also  responsible  for  maintaining 
lists  of  projects  by  programming  avenue  to  be  accomplished 
by  contract  far  the  base.  Contract  programmers  function  as 
engineers  and  as  liaisons  between  civil  engineering  and 
other  base  organizations. 


Introduction 


This  chapter  describes  thB  approach  and  techniques  used 
to  answer  the  research  objective  stated  in  Chapter  I:  Uhat 
general  background  information  do  current  Air  Force  contract 
programmers  in  the  field  perceive  should  be  included  in  a 
handbook  designed  to  help  new  personnel  learn  the  job  of 
contract  programming  as  quickly  as  possible? 

Population 

The  population  of  interest  in  this  research  consisted 
of  all  base  level  and  MAJCOM  level  civil  engineering 
contract  programmers  stationed  in  the  continental  United 
States  (CONUS) .  This  population  consisted  of  military 
personnel  in  the  grades  of  E-7  to  0-5  and  civilian  personnel 
in  the  grades  oF  GS-7  to  Grt-14.  The  exact  number  of  each  of 
these  types  of  personnel  was  not  readily  available  because 
the  job  of  contract  programming  does  not  have  a  unique 
military  Air  Force  Specialty  Code  CAFSC)  nor  a  unique 
civilian  numerical  identifier.  Therefore,  the  population 
was  determined  by  survey . 

A  letter  of  request  was  mailed  to  each  of  the  Air  Force 
bases  in  the  CONUS.  Letters  were  addressed  to  the  Base 
Civil  Engineering  squadron  commander  at  each  base  and  the 
Deputy  Chief  of  Staff  for  Engineering  and  Services  of  each 
of  the  major  commands  headquartered  in  the  CONUS,  requesting 


that  they  provide  the  names  of  the  contract  programmers  on 
their  staffs.  Df  the  89  letters  mailed  out,  82  letters  were 
returned,  resulting  in  the  identification  of  61  military  and 
174  civilian  contract  programmers.  The  distribution  of  the 
programmers,  by  MAJCOM  level  and  base  level,  is  presented  in 
Appendix  A. 

Survey  Instrument 

The  survey  instrument  Ca  questionnaire)  used  in  the 
research  uias  devised  by  the  author  using  knouledge  gained 
from  a  review  of  the  current  literature  C1980  to  1985)  about 
questionnaires.  Using  his  own  experience  as  a  contract 
programmer,  he  determined  what  specific  information  should 
be  included  in  the  questionnaire.  A  review  of  the 
literature  concerning  the  definition,  design,  and 
construction  of  a  questionnaire  is  presented  in  Appendix  B. 

The  author  pretested  the  questionnaire  using  personnel 
at  the  AFIT  School  of  Civil  Engineering.  The  questionnaire 
was  administered  to  the  34  personnel  enrolled  in  the  Project 
Programming  course  CflGT  023)  which  was  held  during  the 
period  3  Feb  86  to  12  Feb  86.  This  course  is  available  to 
and  attended  by  contract  programmers  from  throughout  the  Air 
Force.  The  results  of  the  pretest  indicated  the 
questionnaire  was  adequate  for  its  intended  purpose  and  did 
not  require  any  revisions. 


The  questionnaire  was  then  submitted  to  the  Air  Force 
Manpower  and  Personnel  Center  CAFMPC)  at  Randolph  Air  Force 


Base,  Texas  for  Air  Force  approval.  AFMPC  forwarded  a  copy 
of  the  questionnaire  to  the  Office  of  Civilian  Personnel 
Operations  COCPO),  also  located  at  Randolph  AFB  Capproval  of 
OCPO  is  required  when  surveying  Air  Force  civilians). 

Written  notification  of  approval  from  both  organizations  was 
received  on  B  April  1SB5.  However,  full  approval  was 
contingent  upon  two  recommendations. 

The  first  recommendation  came  from  OCPO  and  required 
the  author  to  submit  information  copies  of  the  questionnaire 
and  approval  letter  to  the  base  level  Civilian  Personnel 
Labor  Management  Relations  office  at  each  base  to  be 
surveyed.  The  second  recommendation  came  from  AFMPC  and 
required  the  author  to  change  the  rating  scale  used  in  the 
questionnaire.  Both  of  these  recommendations  werB 
accomplished  and  the  final  survey  questionnaire  was 
prepared.  This  questionnaire  is  presented  in  Appendix  C. 

Ba£a  -CflllBg&lflD 

The  questionnaires  WBre  mailed  out  to  the  military 
contract  programmers  and  civilian  personnel  offices 
concurrently.  Ten  working  days  were  allowed  for  the 
civilian  personnel  offices  to  respond  to  the  questionnaire 
should  they  have  any  questions  or  problems.  Five  civilian 
personnel  offices  responded  with  telephone  calls  to  the 
author.  Their  questions  were  easily  resolved.  However,  a 
sixth  civilian  personnel  office,  located  at  Kirtland  AFB  New 
Mexico,  disapproved  by  written  notification  the 


administering  of  the  questionnaire  to  any  of  the  civilian 
contract  programmers  on  that  base.  Therefore,  the  six 
personnel  at  that  base  mere  not  surveyed.  The 
questionnaires  for  the  rest  of  the  civilian  personnel  mere 
mailed  out  at  the  end  of  the  ten-day  period. 

Survey  Response 

Two  hundred  twenty  ninB  surveys  were  mailed  to  the 
population  during  the  period  5  May  1SB6  to  30  May  1SB6.  By 
the  cutoff  date  of  IB  June  1986,  19B  surveys  had  been 
returned,  for  a  return  rate  of  B6.5?;.  Five  of  these  surveys 
were  invalid  because  they  were  not  properly  filled  out. 

Data  Analysis 

The  information  collected  through  the  questionnaire 
consisted  of  three  parts.  The  first  part  was  demographic 
data,  which  included  the  personnel  status  Ci.e.,  military  or 
civilian),  command  level  Ci.e.,  MAJCOrt  or  base  level),  and 
job  experience  Ci.e.,  time  in  the  job)  of  the  personnel 
surveyed.  The  second  part  consisted  of  the  personal 
opinions  of  the  survey  participants  as  to  their  degree  of 
recommendation  or  nonrecommendation  to  include  information 
items  Ctopic  areas)  listed  in  the  questionnaire  in  a 
handbook.  The  third  part  of  the  questionnaire  allowed  the 
survey  participants  to  add  additional  information  items 
which  they  felt  should  be  included  in  a  handbook. 

The  analysis  of  the  demographic  data  from  Part  I  of  the 
questionnaire  identified  IS  different  types  of  survey 
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participants  (b.q.,  flAJCOM  civilian  with  over  IB  months 
experience;  base  level  military  with  0-6  months  of 
experience,  etc.).  This  information  was  used  to  determine 
some  general  characteristics  of  the  population  of  contract 
programming  personnel  presently  located  at  the  CONUS  based 
Air  Force  installations. 

The  analysis  of  Part  II  of  thB  questionnaire  consisted 
of  three  parts.  Before  describing  these  parts,  two 
assumptions  about  the  data  collected  must  be  addressed. 

There  are  four  levels  of  measurement  for  characterizing 
data:  nominal,  ordinal,  interval,  and  ratio.  These  levels 
determine  the  type  of  statistical  testing  that  can  be  done 
with  the  data  (12:36).  The  data  derived  from  Part  II  of  the 
questionnaire  is  characterized  as  ordinal  level  data  (*i:90). 
The  data  can  be  ranked  according  to  degree  of  recommendation 
or  nonrecommendation;  however,  the  distance  between  the 
degrees  is  not  measured.  If  the  distance  was  measured,  then 
the  level  of  measurement  would  be  interval . 

There  is  disagreement  among  the  behavioral  sciences 
researchers  as  to  whether  or  not  parametric  tests  (i.e., 
difference  of  means)  can  be  conducted  using  ordinal  level 
data  because  the  distance  between  the  rankings  is  not 
measured  (4:B9-90).  For  the  purpose  of  this  research,  the 
distance  C interval)  between  the  rankings  (degrees)  is 
assumed  to  be  equal.  This  allows  the  data  to  be  treated  as 


interval  level  data  and  parametric  tests  are  then  permitted. 
This  assumption  is  supported  by  Nie  (11:6).  He  notes  that 

IB 


’’except  for  extreme  situations,  interval  statistics  can  be 
applied  to  any  ordinal-level  variable”  (11:63. 

The  second  assumption  that  must  be  made  pertains  to  the 
probability  distribution  of  the  data.  The  central  limit 
theorem  al lous  the  researcher  to  assume  that  thB 
distribution  is  normal  provided  the  sample  size  is  greater 
than  30  (3:198-2013.  Since  the  sample  size  for  this 
research  group  was  193,  a  normal  distribution  was  assumed. 

The  first  part  of  the  analysis  of  Part  II  consisted  of 
the  tabulation  of  the  frequencies  of  the  question  responses 
collected  from  the  sample  population.  The  arithmetic  or 
sample  mean  of  the  frequencies  for  each  question  response 
was  then  calculated  (7:163. 

The  sample  mean  of  the  frequencies  was  calculated  by 
multiplying  the  number  of  responses  for  each  question  by 
their  respective  scale  values  (1,2, 3, 4  or  53,  summing  these 
results,  and  dividing  the  sum  by  the  total  number  of 
responses  for  that  question. 

The  questions  (topic  areas 3  were  then  listed  in  order 
of  emphasis  (rank  ordered 3  from  most  "strongly  recommended” 
to  least  "recommended”  as  determined  by  the  value  of  their 
respective  means.  The  purpose  of  the  rank  ordering  was  to 
determine  which  topic  areas  the  survey  respondents 
considered  the  most  important  for  the  new  contract 
programmer  to  learn. 

Finally,  the  arithmetic  mean  and  standard  deviation  of 
the  sample  means  ware  calculated.  The  purpose  of  this 


analysis  uias  to  determine  whether  or  not  the  sample  mean  of 
any  of  the  topic  areas  was  significantly  different  from  the 
arithmetic  mean  of  the  sample  means.  A  sample  mean  was 
considered  significantly  different  if  its  value  varied  from 
the  arithmetic  mean  value  of  the  sample  means  by  plus  or 
minus  two  or  mors  standard  deviations.  This  measurement  was 
chosen  because  the  probability  of  it  occurring  is  .05. 
Therefore,  its  occurrence  would  imply  that  the  topic  area 
with  that  sample  mean  is  either  recommended  significantly 
higher  or  lower  than  the  rest  of  the  topic  areas. 

The  second  part  of  the  analysis  involved  dividing  the 
population  into  two  groups:  personnel  with  less  than  one 
year  of  experience  in  the  job  and  personnel  with  more  than 
one  year  of  experience.  Tabulation  of  frequencies, 
arithmetic  means,  and  order  of  emphasis  listings  were  again 
accomplished — this  time  by  group  instead  of  by  total  sample 
population.  The  purpose  of  this  analysis  was  to  determine 
whether  or  not  job  experience  affected  the  order  of  emphasis 
placed  on  the  topic  areas. 

Next,  a  "Z”  test  for  the  "difference  of  sample  means” 
was  conducted  (7:171).  This  test  determines  whether  or  not 
there  is  a  statistically  significant  difference  between  the 
sample  mean  of  a  topic  area  on  one  list  versus  the  sample 
mean  of  the  same  topic  area  on  another  list.  A  significance 
level  (probability  of  determining  that  there  is  a  difference 
between  the  means  when  in  reality  there  is  not)  of  .05  was 
chosen  (3:535).  This  level  allows  for  55  percent  confidence 


in  the  accuracy  of  the  test  results.  The  results  of  this 
test  determined  whether  or  not  there  was  a  significant 
difference  between  the  two  groups  in  the  degree  of 
recommendation  placed  upon  each  topic  area. 

The  third  part  of  the  analysis  involved  the  same 
procedures  as  the  second  part,  but  the  total  sample 
population  was  divided  into  two  different  groups:  HAJCOtl 
personnel  and  base  level  personnel .  The  purpose  of  this 
analysis  was  to  determine  whether  or  not  the  level  of 
command  affected  the  order  of  emphasis  placed  upon  the  topic 
areas.  The  analysis  also  determined  whether  or  not  there 
was  a  significant  difference  (difference  of  sample  means 
using  a  significance  level  of  .05)  between  the  two  groups  in 
the  degree  of  recommendation  placed  upon  each  topic  area. 

The  analysis  of  Part  III  of  the  questionnaire  involved 
listing  the  suggested  additional  topic  areas  and  tabulating 
the  frequency  of  responses  for  each  area.  If  30  or  more  of 
the  153  respondents  listed  the  topic  area,  then  it  was 
Included  in  the  recommended  list  of  topic  areas  for  the 
handbook.  This  figure  was  chosen  by  the  author  after 
evaluating  the  list  of  suggested  additional  topic  areas  and 
their  frequencies  and  determining  a  significant  break  point 
in  the  distribution  of  the  frequencies  (10:57).  Those 
additional  topic  areas  not  chosen  to  be  included  in  the 
handbook  were  evaluated  using  content  analysis. 

Content  analysis  was  used  to  determine  whether  or  not 
the  suggested  additional  topic  areas  could  be  grouped 


together  according  to  similar  content  and  either  considered 
additions  to  the  topic  areas  already  identified  in  the 
questionnaire  or  formed  into  fewer  but  larger  topic  areas. 
Content  analysis  was  accomplished  using  the  researcher's 
Judgment  and  was  based  on  the  researcher’s  knowledge  of  the 
subject  C10:B0). 
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IU .  Results  and  Analysis 


Introduction 

This  chapter  presents  the  results  of  thB  survey.  It 
begins  with  a  discussion  of  the  demographic  profile  of  the 
population  as  determined  by  the  participants’  responses  to 
Part  I  of  the  survey .  Several  analyses  are  then  presented 
to  shout  a  comparison  of  the  responses  of  thB  various  types 
of  participants  to  Part  II  of  the  survey.  Finally,  an 
analysis  of  the  open-ended  response  portion  of  the  survey 
CPart  III)  is  presented. 

Demographics 

Part  I  of  the  survey  requested  that  the  participants 
provide  demographic  data  about  themselves.  The  results  of 
this  part  of  the  survey  are  presented  in  Table  I . 

Table  I  shouts  the  population  separated  into  two  large 
categories;  major  command  personnel  CMAJCDH)  and  base  level 
personnel .  These  categories  are  then  divided  into  civilian 
and  military  personnel.  Finally,  the  population  is  further 
categorized  according  to  the  participants'  job  experience. 

Closed-ended  Response  Analysis 

Part  II  of  the  survey  consisted  of  a  closed-ended 
response  format.  It  asked  the  participants  to  provide  their 
opinions  about  whether  or  not  the  particular  topic  areas 
listed  in  the  questionnaire  should  be  included  in  a 


TABLE  I 


Population  Characteristics 


Experience 

majcdm 

Civilian  Military 

Bass. 

Civilian 

Level 

Military 

IB  months  &  over 

24 

4 

71 

12 

12  -  18  months 

2 

2 

B 

9 

6-12  months 

2 

3 

16 

10 

0-6  months 

1 

3 

17 

9 

Total 

29 

12 

112 

40 

handbook.  The  results  of  this  part  of  the  survey  are 
presented  in  Table  II. 

As  the  table  shows,  the  majority  of  the  survey 
participants  at  least  "recommended”  (frequency  of  response 
for  scale  value  of  1  or  5  greater  than  frequency  of  response 
for  any  other  scale  value)  the  inclusion  of  all  14  topic 
areas  listed  in  the  questionnaire  in  a  handbook. 

The  analysis  of  the  results  began  with  the 
determination  of  the  order  of  emphasis  Crank  order)  of  the 
topic  areas .  The  order ,  as  determined  by  the  means  of  the 
frequencies  of  responses,  is  presented  in  Table  III.  Next, 
the  standard  deviation  and  arithmetic  mean  of  the  mean 
values  listed  in  Table  III  were  calculated.  These  values 
are  .319  and  4.197  respectively.  None  of  the  mean  values  of 
the  responses  listed  in  Table  III  varied  from  the  value 
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TABLE  II 


Response  bu  Total  Population 
(Frequencies! 


Topic  Area  * 

(11 

(21 

(31 

(41 

(51 

Air  Force  Regulations 

3 

1 

5 

67 

117 

Airfield,  Noise  and 

Explosive  Safety  Waivers 

2 

11 

41 

B9 

50 

Approval  Authorities 

2 

1 

5 

45 

140 

Base  Master  Planning 

3 

5 

34 

110 

41 

Civil  Engineering  Contract 

Reporting  System  (CEC0RS1 

1 

3 

9 

63 

117 

Construction  Cost  Estimating 

1 

B 

10 

85 

B9 

Economic  Analysis 

1 

B 

22 

9B 

64 

Environmental  Planning 

3 

B 

33 

120 

29 

Facilities  Board 

1 

1 

7 

07 

97 

Planning,  Programming,  and 

Budgeting  System 

4 

14 

2B 

BS 

5B 

Programming  Avenues 

1 

2 

2 

46 

142 

Project  Books 

4 

11 

34 

36 

40 

Project  Identification 

1 

3 

16 

9B 

69 

Real  Estate 

1 

14 

36 

114 

2B 

•  SCALE:  (11  Strongly  do  not  recommend 
(21  Do  not  recommend 
(31  Don’t  knoui/No  opinion 
(41  Recommend 
(51  Strongly  recommend 

Table  III 


Order  of  Emphasis  as  Suggested  by 
the  Total  Population 


Rank 

Topic  Area 

•Mean  Oalue 

1 

Programming  Avenues 

4 .609 

2 

Approval  Authorities 

4 .658 

3 

Air  Force  Regulations 

4.523 

4 

Civil  Engineering  Contract 
Reporting  System  CCECORS) 

4.513 

5 

Facilities  Board 

4.440 

6 

Construction  Cost  Estimating 

4.311 

7 

Project  Identification 

4.166 

B 

Economic  Analysis 

4.119 

9 

Planning,  Programming,  and 
Budgeting  System 

3.94B 

10 

Base  Master  Planning 

3 .936 

11 

Airfield,  Noise  and 

Explosive  Safety  UaivBrs 

3.902 

12 

Project  Books 

3.896 

13 

Environmental  Planning 

3.050 

14 

Real  Estate 

3.790 

•  Mean 

value  is  the  arithmetic  mean  of 

the  responses  as 

determined  by  multiplying  the  number  of  responses  by  their 
respective  scale  values,  summing  these  results,  and  dividing 
the  sum  by  the  total  number  of  responses. 
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of  the  arithmetic  mean  of  the  mean  values  bu  plus  or  minus 
two  or  more  standard  deviations  Ci.e.,  greater  than  4.B35  or 
less  than  3.559  respectively).  Therefore,  none  of  the  topic 
areas  was  recommended  significantly  higher  or  lower  than  the 
other  topic  areas. 

The  next  part  of  the  analysis  involved  dividing  thB 
population  into  various  groups  to  determine  whether  or  not 
the  participants'  Job  experience  or  level  of  command  CflAJCDfl 
or  base  level)  influenced  the  order  of  emphasis  or  degree  of 
recommendation  placed  upon  the  topic  areas. 

First,  the  population  was  categorized  according  to  the 
participants'  job  experience.  The  participants  were  divided 
into  two  groups:  those  with  more  than  one  year  of  experience 
in  the  job  and  those  with  less  than  one  year  of  experience. 
The  distribution  of  their  responses  is  presented  in  Tables 
IU  and  U  respectively.  Each  group  at  least  "recommended” 
the  inclusion  of  all  14  topic  areas. 

The  order  of  emphasis  of  the  topic  areas,  by  group,  is 
presented  in  Tables  UI  and  Mil.  The  order  of  emphasis  was 
very  similar  between  the  two  groups  with  the  exception  of 
three  topic  areas:  Airfield,  Noise,  and  Explosive  Safety 
Ulaivers,  Environmental  Planning,  and  Project  Books. 

Personnel  with  more  than  one  year  of  experience  in  the  job 
ranked  Airfield,  Noise,  and  Explosive  Safety  Ulaivers  higher 
in  emphasis  and  Environmental  Planning  and  Project  Books 
lower  in  emphasis  than  did  the  personnel  with  less  than  one 
year  of  experience  in  the  job. 


TABLE  IU 


RospQnsB  by  Personnel  With  More  Than  1  Year 
of  Experience  in  Job 
(Frequencies) 


Topic  ArBa 

*C1) 

(2) 

(3) 

(4) 

C5 

Air  Force  Regulations 

3 

1 

5 

43 

BO 

Airfield,  Noise  and 

Explosive  Safety  Waivers 

2 

4 

29 

61 

36 

Approval  Authorities 

2 

0 

2 

31 

97 

Base  Master  Planning 

3 

3 

23 

7B 

25 

Civil  Engineering  Contract 
Reporting  System  (CECQRS) 

1 

2 

7 

46 

76 

Construction  Cost  Estimating 

1 

7 

5 

56 

63 

Economic  Analysis 

1 

7 

15 

69 

40 

Environmental  Planning 

3 

7 

23 

79 

20 

Facilities  Board 

1 

1 

6 

61 

63 

Planning,  Programming,  and 
Budgeting  System 

3 

9 

19 

62 

39 

Programming  Avenues 

1 

0 

2 

30 

99 

Project  Books 

4 

9 

21 

69 

29 

Project  Identification 

1 

7 

10 

64 

50 

Real  Estate 

1 

11 

22 

76 

22 

*  SCALE:  (1)  Strongly  do  not  recommend 
C 2)  Do  not  recommend 

(3)  Don’t  knou/No  opinion 

(4)  Recommend 

CS)  Strongly  recommend 


TABLE  U 


Response  by  Personnel  With  Less  Than  1  Year 
of  Experience  in  Job 
(Frequencies) 


Topic  Area 

*C1) 

C2) 

(3) 

(4) 

C5) 

Air  Force  Regulations 

0 

0 

0 

24 

37 

Airfield,  Noise  and 

Explosive  Safety  Waivers 

0 

7 

12 

28 

14 

Approval  Authorities 

0 

1 

3 

14 

43 

Base  Master  Planning 

0 

2 

11 

32 

16 

Civil  Engineering  Contract 
Reporting  System  (CECQRS) 

0 

1 

2 

17 

41 

Construction  Cost  Estimating 

0 

1 

5 

23 

26 

Economic  Analysis 

0 

1 

7 

23 

24 

Environmental  Planning 

0 

1 

10 

41 

3 

Facilities  Board 

0 

0 

1 

26 

34 

Planning,  Programming,  and 
Budgeting  System 

1 

5 

3 

27 

13 

Programming  Avenues 

0 

2 

0 

16 

43 

Project  Books 

0 

2 

13 

27 

13 

Project  Identification 

0 

2 

6 

34 

13 

Real  Estate 

0 

3 

14 

38 

6 

*  SCALE:  Cl)  Strongly  do  not  recommend 

(2)  Do  not  recommend 

(3)  Don’t  knoui/No  opinion 

(4)  Recommend 

(5)  Strongly  recommend 


Table  UI 


Order  of  Emphasis  as  Suggested  by  Personnel 
nore  Than  1  Year  of  Experience  in  Job 

With 

Rank 

Topic  Area 

•Mean 

Ualue 

1 

Programming  Avenues 

4.712 

2 

Approval  Authorities 

4.674 

3 

Air  Force  Regulations 

4 . 4B5 

4 

Civil  Engineering  Contract 

Reporting  System  (CECORS) 

4.470 

5 

Facilities  Board 

4.394 

6 

Construction  Cost  Estimating 

4.311 

7 

Project  Identification 

4.174 

B 

Economic  Analysis 

4.061 

9 

Airfield,  Noise  and 

Explosive  Safety  Waivers 

3.947 

9 

Planning,  Programming,  and 

Budgeting  System 

3.947 

11 

Base  riaster  Planning 

3.902 

ie 

Project  Books 

3.833 

13 

Real  Estate 

3 .011 

14 

Environmental  Planning 

3.B03 

*  Mean  value  Is  the  arithmetic  mean  of  the  responses  as 
determined  bu  multiplying  the  number  of  responses  by  their 
respective  scale  values,  summing  these  results,  and  dividing 
the  sum  by  the  total  number  of  responses. 
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Table  UII 


Order  of  Emphasis  as  Suggested  by  Personnel  With 
Less  Than  1  Year  of  Experience  in  Job 


Rank 

Topic  Area 

•Mean  Ualue 

1 

Programming  Avenues 

4.639 

2 

Approval  Authorities 

4.623 

3 

Air  Force  Regulations 

4.607 

3 

Civil  Engineering  Contract 
Reporting  System  (CECORS) 

4.607 

5 

Facilities  Board 

4.541 

6 

Construction  Cost  Estimating 

4.311 

7 

Economic  Analysis 

4.246 

B 

Project  Identification 

4 . 140 

9 

Project  Books 

4.033 

10 

Base  Master  Planning 

4.016 

11 

Environmental  Planning 

3.951 

11 

Planning,  Programming,  and 
Budgeting  System 

3.951 

13 

Airfield,  Noise  and 

Explosive  Safety  Waivers 

3 .803 

14 

Real  Estate 

3.770 

•  Mean 

value  is  the  arithmetic  mean  of 

the  responses  as 

determined  by  multiplying  the  number  of  responses  by  their 
respective  scale  values,  summing  these  results,  and  dividing 
the  sum  by  the  total  number  of  responses. 


The  standard  deviation  and  arithmetic  mean  of  the  mean 
values  listed  in  Table  U1  mere  calculated  to  be  .324  and 
4.180  respectively.  The  values  for  Table  Oil  mere 
calculated  to  be  .307  and  4.191  respectively.  The  range  of 
values  (arithmetic  mean  plus  or  minus  tuo  standard 
deviations)  uas  calculated  to  be  3.532  to  4.828  for  Table  01 
and  3.577  to  4.805  for  TablB  Oil.  In  neither  case  did  any 
of  the  mean  values  listed  in  the  tables  exceed  the  range  of 
values  for  their  respective  table.  Therefore,  neither  group 
recommended  any  topic  area  significantly  higher  or  louier 
than  the  other  topic  areas. 

The  "difference  of  sample  means"  test  for  the  tuo 
groups,  by  topic  area,  revealed  that  there  uas  no 
significant  difference  in  the  degree  of  recommendation 
betueen  the  groups.  These  results  are  presented  in  Table 
011 I.  An  explanation  of  hou  these  figures  uere  obtained  is 
presented  in  Appendix  0. 

Next,  the  total  population  uas  categorized  according  to 
the  participants’  level  of  command.  The  participants  uere 
divided  into  tuo  groups:  hAJCOn  personnel  and  base  level 
personnel .  The  distribution  of  their  responses  is  presented 
in  Tables  IX  and  X  respectively.  Each  group  at  least 
"recommended"  the  inclusion  of  all  14  topic  areas. 

The  order  of  emphasis  of  the  topic  areas,  by  group,  is 
presented  in  Tables  XI  and  XII.  The  order  of  emphasis  uas 
very  similar  betueen  the  tuo  groups  uith  the  exception  of 
three  topic  areas*.  Construction  Cost  Estimating,  Planning, 


TABLE  VIII 


Difference  of  Semple  Means  Test  Between 
Levels  of  Job  Experience 


Topic  Area 

*  Mean 

Values 

**  Z  Value 

Air  Force  Regulations 

4.4B5 

4.607 

-1 .29 

Airfield,  Noise  and 

Explosive  Safety  Waivers 

3. 347 

3.B03 

1 .02 

Approval  Authorities 

4.674 

4.623 

.50 

Base  Master  Planning 

3 .  SOS 

4.016 

-.95 

Civil  Engineering  Contract 
Reporting  System  CCECQRS) 

4.470 

4.607 

-1.32 

Construction  Cost  Estimating 

4.311 

4.311 

0.00 

Economic  Analysis 

4.061 

4.246 

-1 .57 

Environmental  Planning 

3.B03 

3.951 

-1.37 

Facilities  Board 

4.394 

4.541 

-1.62 

Planning,  Programming,  and 
Budgeting  System 

3.347 

3.951 

-.03 

Programming  Avenues 

4.71S 

4.639 

.74 

Project  Books 

3.B33 

4.033 

-1.50 

Project  Identification 

4.174 

4.14B 

.22 

Real  Estate 

3 .611 

3.770 

.36 

*  Mean  Values  from  Tables  VI  and  VII  respectively . 

••  The  difference  between  sample  means  is  considered 
significant  if  the  Z  Value  is  greater  than  1.36  or  less  than 


Table  IX 


Response  by  MAJCOM  Personnel 
(Frequencies) 


Topic  Area 

*C1) 

C2) 

(3) 

(4) 

(5; 

Air  Force  Regulations 

0 

0 

3 

15 

23 

Airfield,  Noise  and 

Explosive  Safety  Waivers 

1 

1 

11 

21 

7 

Approval  Authorities 

0 

0 

1 

11 

29 

Base  Master  Planning 

1 

0 

8 

25 

6 

Civil  Engineering  Contract 
Reporting  System  (CECQRS) 

0 

1 

4 

16 

20 

Construction  Cost  Estimating 

0 

5 

2 

23 

11 

Economic  Analysis 

0 

2 

5 

24 

10 

Environmental  Planning 

1 

2 

9 

25 

3 

Facilities  Board 

0 

0 

1 

22 

IB 

Planning,  Programming,  and 
Budgeting  System 

0 

4 

4 

17 

15 

Programming  Avenues 

0 

0 

0 

10 

31 

Project  Books 

1 

5 

9 

21 

5 

Project  Identification 

0 

1 

4 

25 

11 

Real  Estate 

0 

6 

9 

22 

4 

*  SCALE:  Cl)  Strongly  do  not  recommend 

(2)  Do  not  recommend 

(3)  Don’t  know/No  opinion 

(4)  Recommend 

(5)  Strongly  recommend 


TABLE  X 


Response  by  Base  Level  Personnel 
(Frequencies) 


Topic  Area 

•Cl) 

(2) 

(3) 

C4) 

C5 

Air  Force  Regulations 

3 

1 

2 

52 

34 

Airfield,  Noise  and 

Explosive  Safety  Waivers 

1 

10 

30 

6B 

43 

Approval  Authorities 

2 

1 

4 

34 

111 

Base  Master  Planning 

2 

5 

26 

84 

35 

Civil  Engineering  Contract 
Reporting  System  (CECQRS) 

1 

2 

5 

47 

97 

Construction  Cost  Estimating 

1 

3 

B 

62 

78 

Economic  Analysis 

1 

B 

17 

74 

54 

Environmental  Planning 

2 

5 

24 

34 

26 

Facilities  Board 

1 

1 

6 

65 

79 

Planning,  Programming,  and 
Budgeting  System 

4 

10 

24 

72 

42 

Programming  Avenues 

1 

2 

2 

36 

111 

Project  Books 

3 

6 

25 

75 

43 

Project  Identif icetion 

1 

B 

12 

73 

58 

Real  Estate 

1 

B 

27 

92 

24 

*  SCALE:  Cl)  Strongly  do  not  recommend 

(2)  Oo  not  recommend 

(3)  Don’t  knaw/No  opinion 

(4)  Recommend 

(5)  Strongly  recommend 


Table  XI 


Order  of  Emphasis  as  Suggested  bg 
MAJCOM  Personnel 


Rank 

Topic  Area 

•Mean  Ualue 

1 

Programming  AvBnues 

4 .756 

E 

Approval  Authorities 

4 .603 

3 

Air  Force  Regulations 

4 .408 

4 

Facilities  Board 

4.415 

5 

Civil  Engineering  Contract 
Reporting  System  CCECQRS) 

4.341 

6 

Project  Identification 

4.1SE 

7 

Planning,  Programming,  and 
Budgeting  System 

4.09B 

B 

Economic  Analysis 

4.0E4 

9 

Construction  Cast  Estimating 

3.976 

10 

Base  Master  Planning 

3.B7B 

11 

Airfield,  Noise  and 

Explosive  Safety  Ulaivers 

3.700 

IE 

Environmental  Planning 

3.603 

13 

Project  Books 

3.5B5 

13 

Real  Estate 

3.505 

•  Mean 

value  is  the  arithmetic  mean  of 

the  responses  as 

determined  bg  multiplying  the  number  of  responses  bg  their 
respective  scale  values,  summing  these  results,  and  dividing 
the  sum  bg  the  total  number  of  responses. 


Table  XII 


Order  of  Emphasis  as  Suggested  by 
Base  Level  Personnel 


Rank 

Topic  Area 

•Mean  Ualue 

1 

Programming  Avenues 

4.671 

2 

Approval  Authorities 

4.651 

3 

Civil  Engineering  Contract 

Reporting  System  CCECORS) 

4.559 

4 

Air  Force  Regulations 

4.533 

5 

Facilities  Board 

4.447 

6 

Construction  Cost  Estimating 

4.401 

7 

Project  Identification 

4.170 

B 

Economic  Analysis 

4.145 

9 

Project  Books 

3.9B0 

10 

Base  Master  Planning 

3.954 

11 

Airfield,  Noise  and 

Explosive  Safety  Waivers 

3.934 

12 

Planning,  Programming,  and 

Budgeting  System 

3.909 

13 

Environmental  Planning 

3 . 895 

14 

Real  Estate 

3 .055 

*  Mean  value  is  the  arithmetic  mean  of  the  responses  as 
determined  by  multiplying  the  number  of  responses  by  their 
respective  scale  values,  summing  these  results,  and  dividing 
the  sum  by  the  total  number  of  responses. 


< 

I 

* 

I 

I  Programming,  and  Budgeting  System,  and  Project  Books. 

| 

flAJCQfl  personnel  ranked  Planning,  Programming,  and  Budgeting 
System  higher  in  emphasis  and  Construction  Cost  Estimating 
and  Project  Books  lower  in  emphasis  than  did  the  base  level 
personnel . 

The  standard  deviation  and  arithmetic  mean  of  the  mean 
values  listed  in  Table  XI  were  calculated  to  be  .368  and 
4.101  respectively.  The  values  for  Table  XII  were 
calculated  to  be  .30S  and  4. BSE  respectively.  The  range  of 
values  (arithmetic  mean  plus  or  minus  two  standard 
deviations)  was  calculated  to  be  3.3E5  to  4.677  for  Table  XI 
and  3.604  to  4.840  for  Table  XII.  In  neither  case  did  any 
of  the  mean  values  listed  in  the  tables  exceed  the  range  of 
values  for  their  respective  table.  Therefore  neither  group 
recommended  any  topic  area  significantly  higher  or  lower 
than  the  other  topic  areas. 

The  "difference  of  sample  means"  test  for  thB  two 
groups,  by  topic  area,  revealed  that  there  was  a  significant 
difference  in  the  degree  of  recommendation  between  the 
groups  concerning  Construction  Cost  Estimating  and  Project 
Books.  These  results  are  presented  in  Table  XIII. 

Open-ended  Response  Analysis 

Part  III  of  the  survey  consisted  of  an  open-ended 


response  format.  It  asked  the  participants  to  provide  and 
rate,  as  to  degree  of  recommendation,  additional  topic  areas 
which  they  thought  should  be  included  in  the  handbook  but 


TABLE  XIII 

Difference  of  Sample  Means  Test  Between 
Levels  of  Command 


Topic  Area 

•  Mean 

Ualues 

**  2  Ualue 

Air  Force  Regulations 

4.4BB 

4.533 

-.33 

Airfield,  Noise  and 

Explosive  Safety  Waivers 

3.7B0 

3.334 

-1 .04 

Approval  Authorities 

4.683 

4.651 

.23 

Base  Master  Planning 

3.B7B 

3.354 

-.51 

Civil  Engineering  Contract 
Reporting  System  CCECORS) 

4.341 

4.553 

-1.68 

Construction  Cost  Estimating 

3.376 

4.401 

***  -2.76 

Economic  Analysis 

4.0B4 

4.145 

-.83 

Environmental  Planning 

3.683 

3.835 

-1.53 

Facilities  Board 

4.415 

4.447 

-.36 

Planning,  Programming,  and 
Budgeting  System 

4.03B 

3.  SOB 

1.14 

Programming  Avenues 

4.756 

4.671 

1 .00 

Project  Books 

3.5B5 

3 .380 

-£.40 

Project  Identification 

4.122 

4.17B 

-.44 

Real  Estate 

3 .565 

3.855 

-1 .81 

*  Mean  Ualues  from  Tables  XI  and  XII  respectively 
••  The  difference  between  sample  means  is  considered 
significant  if  the  Z  Ualue  is  greater  than  1.36  or  less  than 
-1.36. 

Significant  difference 


were  not  already  identified. 

One  hundred  and  one  survey  participants  responded  to 
this  portion  of  the  questionnaire.  The  responses  were  of 
three  types:  additional  topic  areas,  specific  information  to 
be  included  in  the  topic  areas  identified,  and  specific 
guidance  on  how  the  handbook  should  be  written.  The  two 
latter  types  of  responses  are  presented  in  Appendix  E. 

The  suggested  additional  topic  areas  and  their 
associated  frequencies  of  response  are  shown  in  Table  XIU. 
The  first  column  of  the  table  shows  the  suggested  additional 
topic  areas.  The  second  column  shows  the  frequencies  of 
responses  according  to  the  scale  value  (4  or  5)  given  to  the 
topic  area  by  the  respondents.  The  final  column  is  the 
total  number  of  responses  for  the  topic  area.  The  topic 
areas  are  listed  in  decreasing  order  by  total  number  of 
responses.  No  significance  is  implied  by  the  order  of  the 
list  where  topic  areas  with  equal  frequencies  of  response 
are  listed. 

The  first  part  of  the  analysis  of  open-ended  responses 
consisted  of  determining  whether  or  not  any  of  the  suggested 
additional  topic  areas  had  a  frequency  of  response  which  met 
the  criteria  presented  in  Chapter  III  (f. jquency  of  response 
of  30  or  morel.  Only  one  topic  area,  Importance  of  and 
Sample  139171391c  Documents,  met  this  criteria. 

The  second  part  of  the  analysis  was  accomplished  using 
content  analysis.  The  author  evaluated  the  list  of 
additional  topic  areas  and  grouped  those  with  similar 
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TABLE  XIO 


Suggested  Additional  Topic  Areas 
(Frequencies) 


Topic  Area 

•4/5 

Tota  1 

Importance  of  and  sample  13S1/13S1C  documents 

6/29 

35 

Interrelationships  between  contract  programmers 
and  other  sections  in  civil  engineering  squadron 

3/9 

12 

Difference  between  types  of  work  classifications 
(i.e.,  repair,  alteration,  maintenance,  etc.) 

1/10 

11 

nCP  approval  process  (flow  and  timing) 

1/10 

11 

How  to  calculate  and  Justify  space  and/or 
facility  requirements 

2/9 

11 

Interrelationships  between  contract  programmer 
and  project  requestor 

1/7 

B 

Easy  to  read  tables  listing  codes  and  approval 
limits  used  on  1391 ’s  which  could  be  easily 
referenced  by  new  programmers 

0/3 

3 

Program  Objectives  flemarandum  process 

1/2 

3 

List  of  actual  contract  programmer’s 
responsibilities 

2/1 

3 

Site  approval  requirements  and  process 

2/1 

3 

Procurement  process 

3/0 

3 

Explanation  of  OSh  budget  and  relationship  to 
contract  programming 

0/2 

2 

File  system  for  programming  guidance  letters 

0/2 

2 

Wins  use  and  computer  skills 

0/2 

2 

(continued) 


•  SCALE:  (4)  Recommend 

(5)  Strongly  Recommend 


Topic  Area 


*4/5  Total 


PDC  guide  0/E  E 

Canned  requirement,  situation,  and  impact  0/E  E 

statements  for  different  situations 

Public  lau  and  DODIs  dealing  with  DOD  facility  0/E  E 

construction 

Correspondence  procedures:  samples  of  letters,  0/E  E 

messages,  point  papers,  staff  summary  sheets 

Explanation  of  EEIC’s  1/1  E 

Quick  reference  dictionary  of  terms  and  1/1  E 

abbreviations 

What  to  keep  in  files  (project  and  personal!  1/1  E 

Tenant  units,  host-tenant  agreements  1/1  E 

Audit  issues  and  how  to  handle  an  audit  1/1  E 

The  importance  of  a  reliable  source  of  current  0/1  1 

design  status  and  construction  status 

Function  of  category  codes  0/1  1 

Provide  stress  management  classes  0/1  1 

How  to  combat  pressure  to  misclassify  work  in  an  0/1  1 

attempt  to  stay  under  the  SE00.000  minor 
construction  limitation 

How  to  straighten  things  out  after  they  have  0/1  1 

gone  through  system  (  >35?;  design!  and  final 
project  is  entirely  different  than  original  work 
request  both  in  scope  and  type  of  funding 

Base  level  interface  with  flAJCDtt  personnel  0/1  1 


C continued! 


•  SCALE:  (4!  Recommend 

(S!  Strongly  Recommend 


Topic  Area 


•4/5  Total 


Military  career  progression  tie-in  0/1  1 

BCE  responsibility  for  maintaining  non-real  0/1  1 

property  shelters/systems 

Prioritizing  projects  based  on  money  projected  0/1  1 

to  be  available  for  funding 

Recommended  training  For  programmers  0/1  1 

A  job  description  of  an  MCP  programmer  0/1  1 

Hou»  to  present  and  sell  *  project  0/1  1 

Model  Installation  Program  0/1  1 

MAJC0M  technical  assistant  teams  0/1  1 

Communications-electronic  fund  sources  and  0/1  1 

responsibilities 

AF0LDS  retrieval  system  and  how  to  use  it  0/1  1 

Mission  and  organization  of  the  BCE  organization  0/1  1 

Describe  concept  of  "Whole  House  Improvement”  0/1  1 

For  TAC  bases  -  S6  N0U  85  TAC  DE  Letter  0/1  1 

CFile  #  DE-416-B5)  Delegation  of  Project  Approval 
Authority 

HQ  TAC  Guidance  Package  on  MCP  1331  preparation  0/1  1 

C "There’s  No  Room  For  Error”) 

Explanation  of  the  impact  asbestos  regulations  0/1  1 

have  on  contract  programming 

Development  of  work  area  layouts  for  projects  0/1  1 


(continued) 


•  SCALE:  (4)  Recommend 

(5)  Strongly  Recommend 


TABLE  XIU--Contini 


Topic  Area 


*4/5  Total 


Discussion  of  who’s  in  charge  -  the  base  or 
the  riAJCOM 


I 


International  exchange  rates  and  their  affect 
on  program  costs  overseas 

Description  of  Project  Life  Cycle 
Cconception  to  disposal  5 

Difference  in  statuatory  versus  regulatory 
limitations 


& 


Purchase  Requests 

Programming  as  a  service  to  the  customer  (user) 


*  SCALE:  (4)  Recommend 

(51  Strongly  Recommend 


content  together.  The  first  11  groups  contain  responses 
associated  with  topic  areas  listed  in  Part  1 1  of  the 
questionnaire.  The  remaining  three  groups  are  new  topic 
areas.  The  groups  are  presented,  in  no  significant  order, 
as  follows: 


Group  1:  Approval  Authorities 


Difference  in  statuatory  versus  regulatory 
limitations 

For  TAC  bases  -  25  NDU  85  TAC  DE  Letter 
(File  #  DE-41B-B5)  Delegation  of  Project 
Approval  Authority 


Group  2:  Base  Master  Planning 


Site  approval  requirements  and  process 


;tr<:»Viv*V,»vv 


Group  3:  Civil  Engineering  Contract  Reporting  System 

Wins  use  and  computer  skills 

AFOLDS  retrieval  system  and  how  to  use  it 

Group  4:  Economic  Analysis 

Description  of  Project  Life  Cycle 
(conception  to  disposal} 

Group  5:  Environmental  Planning 

Explanation  of  the  impact  asbestos 

regulations  have  on  contract  programming 

Group  6:  Facilities  Board 

Prioritizing  projects  based  on  money 
projected  to  be  available  for  funding 

Group  7:  Planning,  Programming,  and  Budgeting  System 

MCP  approval  process  Cflou  and  timing} 

Program  Objectives  Memorandum  process 
Explanation  of  O&M  budget  and  relationship 
to  contract  programming 
Explanation  of  EEIC’s 

Group  B:  Programming  Avenues 

Difference  between  types  of  uiork 

classifications  Ci.e.,  repair,  alteration, 
maintenance ,  etc , } 

CommunicatiDns-electronic  fund  sources  and 
responsibilities 

Group  9:  Project  Identification 

Interrelationships  between  contract 
programmer  and  project  requestor 

Group  10:  Real  Estate 

Function  of  category  codes 

Group  11:  Importance  of  and  Sample  1391/1391c  Documents 

How  to  calculate  and  justify  space  and/or 
facility  requirements 
Canned  requirement,  situation,  and  impact 
statements  for  different  situations 
HQ  TAC  Guidance  Package  on  MCP  1331 

preparation  C ’’There’s  No  Room  for  Error’’} 


Group  IS:  Base  Civil  Engineering  Organization 

Interrelationships  between  contract 

programmers  and  other  sections  in  civil 
engineering  squadron 
List  of  actual  contract  programmer’s 
responsibilities 

The  importance  of  a  reliable  source  of 
current  design  status  and  construction 
status 

Base  level  interface  with  MAJCQM  personnel 
BCE  responsibility  for  maintaining  non-real 
property  shelters/systems 
A  job  description  of  an  flCP  programmer 
Mission  and  organization  of  the  BCE 
organization 

Discussion  of  who’s  in  charge  -  the  base  or 
the  MAJCQM 

Programming  as  a  service  to  the  customer 
C user  3 

Group  13:  Project  Funding 

Procurement  process 

International  exchange  rates  and  their 
affect  on  program  costs  overseas 
Purchase  Requests 

Group  14:  Personal  Aids  and  Guidance 

Easy  to  read  tables  listing  codes  and 

approval  limits  used  on  1391 ’s  which  could 
be  easily  referenced  by  new  programmers 
File  system  for  programming  guidance  letters 
Correspondence  procedures:  samples  of 
letters,  messages,  point  papers,  staff 
summary  sheets 

Quick  reference  dictionary  of  terms  and 
abbreviations 

tilhat  to  keep  in  files  (project  and  personal) 
Provide  stress  management  classes 
How  to  combat  pressure  to  misclassify  work 
in  an  attempt  to  stay  under  the  SB00.000 
minor  construction  limitation 
How  to  straighten  things  out  after  they  have 
gone  through  system  C  >35^  design)  and 
final  project  is  entirely  different  than 
original  work  request  both  in  scope  and 
type  of  Funding 

Military  career  progression  tie-in 
Recommended  training  for  programmers 
How  to  present  and  sell  a  project 
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The  remaining  eight  suggested  additional  topic  areas  in 
Table  XIU  did  not  appear  to  be  similar  in  content  to  any  of 
the  other  topic  areas  and  therefore  remain  as  single  topic 
areas.  They  are-. 

Tenant  units,  host-tenant  agreements 

Audit  issues  and  how  to  handle  an  audit 

nodel  Installation  Program 

flAJCOfl  technical  assistant  teams 

Describe  concept  of  ’’Whole  House  Improvement” 

Development  of  work  area  layouts  for  projects 

PDC  guide 

Public  laui  and  DODIs  dealing  with  DOD  facility 
construction 


The  objective  of  this  research  urns  to  determine  what 


general  background  information  the  current  Air  Force 
contract  programmers  in  the  field  perceive  should  be 
included  in  a  handbook  designed  to  help  new  personnel  learn 
the  job  of  contract  programming  as  quickly  as  possible. 

This  chapter  will  present  the  conclusions  drawn  from 
the  analyses  performed  on  the  survey  data.  It  will  then 
present  the  author's  recommendations  for  what  general 
information  should  be  included  in  the  handbook  and  why. 

These  recommendations  are  based  on  the  results  and  analysis 
of  the  survey  data.  The  chapter  will  conclude  with 
recommendations  for  future  research. 

Copcluaiaas 

The  questionnaire  return  rate  for  this  research  was 
86.5%.  This  high  rate  of  return  indicates  that  the  interest 
is  high  among  the  contract  programmers  in  the  field 
concerning  the  development  of  a  handbook  for  new  contract 
programmers.  If  the  contract  programmers  had  not  perceived 
a  need  for  the  handbook,  they  probably  would  not  have 
responded  to  the  survey  in  the  manner  that  they  did. 

The  results  for  Part  II  of  the  questionnaire  show  that 
regardless  of  whether  or  not  the  population  was  evaluated  as 
a  whole  or  in  smaller  groups,  the  survey  respondents  at 


least  recommended  the  inclusion  of  all  14  topic  areas  listed 
in  the  questionnaire.  It  appears  then,  that  these  14  topic 
areas  are  the  minimum  number  of  topic  areas  to  be  included 
in  the  handbook . 

The  next  observation  is  that  regardless  of  how  the 
population  was  grouped,  the  five  highest  ranked  topic  areas 
remained  the  same:  Programming  Avenues,  Approval 
Authorities,  Air  Force  Regulations,  Civil  Engineering 
Contract  Reporting  System,  and  Facilities  Board.  It  can  be 
assumed  that  these  topic  areas  are  the  most  important  for 
the  new  contract  programmer  to  learn  and  probably  require 
the  majority  of  his  time  in  the  job. 

The  lowest  rank  ordered  topic  area  Ci.e.,  rank  order  of 
13  or  141 ,  regardless  of  grouping,  was  consistently  Real 
Estate.  This  indicates  that  this  topic  area  does  not  play  a 
major  role  in  the  job  of  contract  programming. 

None  of  the  groups  recommended  any  of  the  topic  areas 
significantly  higher  or  lower  than  the  other  topic  areas. 
Therefore,  the  majority  of  the  topic  areas  should  receive 
approximately  equal  emphasis  in  the  handbook. 

When  the  population  was  separated  into  two  groups 
according  to  their  job  experience  (time  in  the  job),  no 
significant  difference  was  determined  between  the  groups  as 
to  their  degree  of  recommendation  for  any  one  topic  area. 
However,  the  two  groups  did  disagree  on  the  order  of 
emphasis  for  three  topic  areas. 

Personnel  with  more  than  one  year  of  experience  in  the 
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Job  placed  a  greater  emphasis  on  the  need  for  the  topic  area 
Airfield,  Noise,  and  Explosive  Safety  Waivers  than  did  the 
personnel  with  less  than  one  year  of  experience.  These 
types  of  waivers  are  not  required  for  every  project; 
therefore,  it  can  be  assumed  that  personnel  with  less  than 
one  year  of  experience  may  not  have  been  exposed  to  the 
requirement  for  these  waivers  while  personnel  with  more 
experience  probably  have.  Personnel  with  less  experience 
would  therefore  not  sbb  a  need  to  emphasize  the  topic  area 
as  much . 

Personnel  with  more  than  one  year  of  experience  did  not 
place  as  great  an  emphasis  on  two  areas,  Environmental 
Planning  and  Project  Books,  as  did  the  personnel  with  less 
experience.  This  difference  can  perhaps  be  explained  by  the 
fact  that  environmental  planning  and  project  books  are  often 
accomplished  (depending  on  the  base)  by  other  sections 
within  the  BCE  organization.  Thus,  personnel  with  more  than 
one  year  of  experience,  knowing  that  these  areas  may  be 
accomplished  by  other  sections  within  the  BCE  organization, 
would  not  emphasize  them  as  much.  Conversely,  personnel 
with  less  than  one  year  of  experience  apparently  realize  the 
importance  of  these  topic  areas,  but  may  not  realize  who  is 
responsible  for  accomplishing  them. 

When  the  population  was  separated  into  two  different 
groups  according  to  their  level  of  command  CMAJCOfl  or  base 
level),  significant  differences  were  determined  between  the 
groups  as  to  their  degree  of  recommendation  for  two  areas. 


The  two  groups  also  disagreed  on  the  order  of  emphasis  for 
these  two  areas  as  well  as  the  order  of  emphasis  for  a  third 


area . 

Base  level  personnel  placed  a  greater  emphasis  and 
higher  degree  of  recommendation  on  the  need  to  include  the 
topic  areas  of  Construction  Cost  Estimating  and  Project 
Books  than  did  the  MAJCOM  level  personnel .  MAJCOH 
programming  personnel  do  not  normally  accomplish 
construction  cost  estimates  and  project  books.  They  do 
however,  review  the  estimates  and  project  books  submitted  to 
them  by  the  base  level  personnel.  Therefore,  it  stands  to 
reason  that  nAJCOH  personnel  would  not  see  a  need  to 
emphasize  these  areas  to  the  same  degree  as  did  the  base 
level  personnel . 

riAJCOn  personnel  placed  a  greater  emphasis  on  the  need 
to  include  the  topic  area  of  Planning,  Programming,  and 
Budgeting  System  CPPBS3  than  did  the  base  level  personnel. 
This  difference  may  have  been  caused  by  the  wording  on  the 
questionnaire.  The  questionnaire  referred  to  what  PPBS  is 
at  the  national  level .  Base  level  personnel  may  have  keyed 
on  the  words  ’’national  level”  and  did  not  see  a  need  to 
emphasize  such  a  broad  topic.  Another  possible  conclusion 
is  that  the  MAJCOM  personnel  saw  a  greater  need  to  include 
this  topic  area  because  they  are  more  directly  involved  in 
the  entire  PPB5  while  the  base  level  programmers  participate 
primarily  in  the  programming  phase. 

Part  111  of  the  survey  asked  the  respondents  to  provide 
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|  additional  topic  areas  to  be  included  in  the  handbook.  The 

i 

» 

|  results  showed  that  one  topic  area,  Importance  of  and  Sample 

1391/ 1391c  Documents,  had  a  frequency  of  response  (35)  that 
was  considerably  higher  than  all  other  suggested  topic  areas 
(frequency  of  response  of  12  or  less).  Having  met  the 
criteria  contained  in  Chapter  III,  this  topic  area  was 
included  in  the  recommended  list  of  topic  areas  for  the 
handbook.  The  high  frequency  of  response  for  this  one  topic 
area  directly  reflects  the  perceived  importance  placed  on 
this  area  for  the  new  contract  programmer.  As  previously 
stated,  one  of  the  responsibilities  of  the  contract 
programmer  is  to  accomplish  1391/1391C  documents. 

The  results  of  the  content  analysis  performed  on  the 
data  collected  from  Part  III  of  the  survey  showed  that  23  of 
the  suggested  additional  topic  areas  focused  on  three  major 
issues  that  were  not  addressed  in  Part  1 1  of  the  survey.  A 
perceived  need  for  the  new  contract  programmer  to  be  aware 
of  the  roles  and  responsibilities  of  the  entire  base  civil 
engineering  organization  was  clearly  evident.  In  addition, 
an  awareness  of  the  process  of  project  funding  and  the  value 
of  possessing  personal  files  containing  reference  materials 
for  the  job  were  identified. 

BBCommenflatiang 

The  results  of  this  research  have  shown  that  a  handbook 


designed  to  help  new  personnel  learn  the  job  of  contract 
programming  as  quickly  as  possible  is  needed  and  should 


include  at  least  15  topic  areas.  Since  the  responses  to  the 
questionnaire  uerB  based  on  the  descriptions  given  for  each 
topic  area  listed  in  the  questionnaire,  it  is  recommended 
that  these  descriptions  be  used  as  a  guide  when  determining 
what  specific  information  to  put  into  the  handbook. 

The  following  is  the  list  of  the  14  original  topic 
areas  and  their  descriptions  as  stated  on  the  questionnaire. 
The  fifteenth  topic  area  is  listed  with  the  author’s 
recommended  description  for  it. 

1.  Air  Force  Regulations.  A  listing  and  description 
of  the  regulations  and  manuals  applicable  to 
contract  programming . 

2.  Airfield,  Noise,  and  Explosive  Safety  Waivers.  A 
description  of  what  each  of  these  waivers  is, 
their  purpose,  and  why  the  contract  programmer 
must  know  about  them. 

3.  Approval  Authorities.  A  description  of  what 
approval  authorities  are,  their  legal  aspects,  and 
how  they  impact  the  programming  process. 

4.  Base  Master  Planning.  A  description  of  what  base 
master  planning  is,  its  makeup  and  function,  and 
how  the  contract  programmer  interfaces  with  it. 

5.  Civil  Engineering  Contract  Reporting  System 
(CECQRS) .  A  description  of  what  CECORS  is,  its 
purpose,  what  information  it  provides,  and  what 
the  contract  programmer  would  use  the  information 


6.  Construction  Cast  Estimating.  A  description  of 
the  various  methods  of  cost  estimating  Ci.e., 
like  structure,  square  foot  costing,  systems 
costing,  unit  pricing)  and  the  role  of  the 
contract  programmer  in  the  preparation  of  a  cost 
estimate . 

7.  Economic  Analysis.  A  description  of  uihat  an 
economic  analysis  is,  its  purpose,  and  the  role  of 
the  contract  programmer  in  the  preparation  of  one. 

B.  Environmental  Planning.  A  description  of  what 
environmental  planning  is,  its  purpose,  and  how 
the  contract  programmer  interfaces  with  it . 

9.  Facilities  Board.  A  description  of  what  the 
facilities  board  is,  its  makeup  and  function,  and 
the  role  of  the  contract  programmer  in  preparing 
for  one. 

10.  Planning,  Programming,  and  Budgeting  System.  A 
description  of  what  the  planning,  programming,  and 
budgeting  system  at  the  national  level  is,  its 
makeup  and  function,  and  how  the  contract 
programmer  interfaces  with  it. 

11.  Programming  Avenues.  A  description  of  the  various 
programming  avenues  Ci.e.,  Military  Construction 
Program,  Operations  and  Maintenance  Program,  etc.) 
including  the  type  of  work  they  support  and  their 
limitations . 
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12.  Project  Books.  A  description  of  what  project 
books  are,  their  purpose,  and  the  role  of  the 


contract  programmer  in  the  preparation  of  them. 

13.  Project  Identification.  A  description  of  how 
projects  get  identified  and  how  they  are  processed 
before  they  reach  the  contract  programmer. 

14.  Real  Estate.  A  description  of  the  real  estate 
function,  its  purpose,  and  how  the  contract 
programmer  interfaces  with  it. 

15.  Importance  of  and  Sample  13S1/1351C  Documents. 
Explanation  of  the  importance  of  the  DD  Form  1391 
programming  document  and  examples  of  properly 
completed  □□  Forms  1391/1391c  for  thB  various 
programming  avenues  along  with  an  explanation  of 
the  sources  of  the  information  found  on  the 
document . 

The  author  recommends  that  the  handbook  be  written  by 
someone  with  at  least  one  year  of  experience  in  contract 
programming.  This  minimum  level  of  experience  is  necessary 
for  proper  evaluation  of  the  other  suggested  topic  areas  not 
considered  for  inclusion  by  the  author  of  this  research. 

Recommendations  for  Future  Research 


This  research  was  intended  to  be  the  initial  step 
towards  the  publication  of  a  handbook  for  new  contract 
programmers.  Recommendations  for  future  research  include 
the  fallowing: 
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1 .  Survey  the  Headquarters  United  States  Air  Force 


contract  programming  personnel . 

2 .  Survey  the  contract  programmers  stationed  overseas . 

3.  Research  the  job  of  the  contract  programmer  in  a 
wartime  environment. 

4.  Write  and  publish  the  handbook. 


Appendix  A:  Survey  Population 


flAJCOfl  Level  Number  of 
List  Programmers 

Headquarters  Air  Force  Logistics  Command  5 
Headquarters  Air  Force  Systems  Command  6 
Headquarters  Air  Training  Command  5 
Headquarters  dilitary  Airlift  Command  5 
Headquarters  Strategic  Air  Command  7 
Headquarters  Tactical  Air  Command  *17 


Subtotal  CdAJCOn  Level!  45 


Base  Level 
List 

Altus  OK  1 
Andrews  MD  4 
Barksdale  LA  3 
Beale  CA  2 
Bergstrom  TX  3 
Blytheville  AR  1 
Brooks  TX  1 
Cannon  Nfl  2 


*  Includes  Requirements  and  Program  Development  Personnel. 
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Base  Level 
List 


Number  of 
Programmers 


Carswell  TX  3 
Castle  CA  2 
Chanute  IL  *4 
Charleston  SC  1 
Cheyenne  Mountain  Complex  CD  1 
Columbus  MS  2 
Davis  Monthan  AZ  2 
Dover  DE  2 
Dyess  TX  2 
Edwards  CA  3 
Eglin  FL  2 
Ellsworth  SD  1 
England  LA  2 
F.E.  Ularren  WY  1 
Fairchild  WA  2 
Soodfellow  TX  1 
Grand  Forks  ND  2 
Griffiss  NY  1 
Grissom  IN  3 
Hanscom  MA  3 
Hill  UT  6 
Holloman  NM  *4 
Homestead  FL  £ 
K . I .  Sawyer  MI  3 

3 


SB 


Kessler  MS 


Base  Level 
List 


Number  of 
Programmers 


Kelly  TX  1 
Kirtland  NM  6 
Lake  Head  NO  1 
Laughlin  TX  2 
Little  Rock  AR  4 
Loring  ME  5 
Lowry  CD  1 
Luke  A2  2 
nacDill  FL  6 
flalmstrom  NT  2 
Narch  CA  2 
Mather  CA  1 
Maxwell  AL  2 
NcChord  U1A  4 
McClellan  CA  2 
McConnell  KS  2 
Minot  ND  1 
Moody  GA  3 
Mountain  Home  ID  1 
Myrtle  Beach  SC  2 
Nellis  NO  3 
Norton  CA  2 


Offutt  NB 
Patrick  FL 

Paasa  NH 


4 

3 

2 
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Base  Level 
List 


Number  of 
Programmers 
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Peterson  CO 
Plattsburg  NY 
Pope  NC 
Reese  TX 
Robins  GA 
Scott  IL 

Seymour  Johnson  NC 
Shsu  SC 
Sheppard  TX 
Tinker  OK 
Travis  CA 
Tyndall  FL 
USAF  Academy  CO 
Uandenberg  CA 
Whiteman  MO 
Williams  AZ 
Wright-Patterson  OH 
Wurtsmith  (II 

Subtotal  (Base  Level) 


riAJCOM  Level  45 
Base  Level  ISO 

Total  S35 


Appendix  B:  Discussion  On  Questionnaires 
Introduction 

Thia  discussion  provides  a  review  of  the  current 
literature  (1SB0  to  1385)  written  about  questionnaires, 
dore  specifically,  this  discussion  will  define  what  a 
questionnaire  is  and  describe  some  of  the  advantages  and 
disadvantages  of  using  questionnaires.  Furthermore,  the 
design  and  construction  of  questionnaires  and  a  significant 
problem  with  the  use  of  questionnaires  will  be  discussed. 

Definition 

A  questionnaire  is  a  survey  instrument  used  to  collect 
data  by  presenting  a  list  of  questions  or  statements  to 
possible  respondents.  The  recipients  of  the  questionnaire 
are  asked  to  respond  to  the  questions  and  statements,  and 
their  responses  Cdata)  are  then  collected  and  analyzed. 

Advantages  and  Disadvantages  of  Questionnaires 

According  to  Weiss,  "Quest ionna ires  are  an  excellent 
way  of  obtaining  large  amounts  of  information  from  a  large 
number  of  people  at  a  relatively  low  cost”  (14:7).  Weiss 
C  1*4 : 7-8)  presents  both  advantages  and  disadvantages  in  using 
questionnaires  over  other  survey  instruments  for  data 
collection . 

The  first  advantage,  when  comparing  the  use  of 
questionnaires  to  the  direct  personal  interview  technique, 


is  that  "The  possibility  that  the  interviewer  might  directly 
influence  an  answer  is  avoided"  (14:7).  According  to  Emory, 
the  interviewer  can  "distort  the  results  of  any  survey  by 
inappropriate  suggestions,  word  emphasis,  tone  of  voice,  and 
question  rephrasing”  (4:167).  This  possibility  is  minimized 
by  using  a  questionnaire. 

Another  advantage  is  that  "The  persons  answering  the 
questionnaire  have  more  time  to  think  about  and  investigate 
a  question  before  answering  it”  (14:7).  This  allows  the 
individuals  the  opportunity  to  evaluate  their  answer  before 
responding . 

The  third  advantage  is  that  "Questionnaires  solve  the 
problem  of  geographical  limitations”  (14:7).  The 
questionnaire  can  be  mailed  out  within  largB  geographical 
areas,  allowing  the  researcher  to  contact  a  broad  base  of 
respondents.  Also,  "Responders  are  often  more  accessible  by 
mail  than  they  are  far  a  personal  interview”  (14:7). 
Questionnaires  allow  the  interviBWBr  to  get  information  from 
people  he  may  not  otherwise  be  able  to  reach. 

Another  advantage  is  that  "The  time  devoted  by  the 
interviewer  to  the  survey  is  considerably  less,  especially 
when  many  people  are  interviewed"  (14:7).  Therefore,  "The 
cost  of  the  questionnaire  is  low  compared  to  the  cost  of 
numerous  personal  interviews”  (14:7). 

The  final  advantage  presented  by  Uleiss  is  that 
"Questionnaires  are  faster  when  a  larger  number  of  people 
are  involved  because  they  Cthe  people)  can  be  handled 
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simultaneously”  C14:7). 


The  advantages  described  above  must  be  weighed  against 
the  disadvantages  of  questionnaires,  some  of  which  are 
listed  below. 

The  first  disadvantage  is  that  "The  receivers  of 
questionnaires  who  feel  strongly  about  the  subject  are  more 
likely  to  respond  than  those  who  have  minor  or  no  interest” 

C 14 : 8) .  This  fact  can  result  in  responses  which  only 
present  one  side  of  the  issue. 

Another  disadvantage  is  that  “Questionnaires  lack 
persuasiveness  in  getting  responders  to  complete  and  answer 
every  question”  C14:8).  The  responder  may  ignore  or 
overlook  questions.  This  can  be  a  disadvantage  if  responses 
to  all  the  questions  on  the  questionnaire  are  needed  to 
create  meaningful  results  from  the  survey. 

□ne  final  disadvantage  presented  by  Ueiss  is  that  "You 
do  not  get  100  percent  return  of  questionnaires"  C14:B). 

This  can  be  a  problem  if  the  sample  size  was  small  to  begin 
with.  If  the  return  rate  of  the  questionnaires  is  not 
adequate,  as  determined  by  the  researcher,  then  the  results 
of  the  survey  may  lack  credibility  or  may  be  inappropriate 
or  impassible  to  analyze. 

Design  of  Questionnaires 

The  design  of  the  questionnaire  has  a  large  impact  on 
the  usefulness  and  value  of  the  information  obtained.  "The 
usefulness  of  the  information  [collected  from  the 
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questionnaire]  depends  upon  the  clarity  of  the  questions  and 

upon  thorough  investigation”  (14:7).  Also  ’’questionnaires 

are  only  valuable  if  they  ask  appropriate  questions  uhich 

people  can  understand  and  ask  them  in  a  format  people  can 

answer”  (9:1003.  Therefore,  the  design  of  the  questionnaire 

must  address  the  clarity  and  appropriateness  of  the 

questions  as  well  as  the  format  of  the  questions. 

Maher  (9:100-104)  breaks  the  process  of  questionnaire 

design  into  three  areas.  These  areas  are  as  follows; 

First,  can  a  standardized  questionnaire  be  used, 
or  should  an  original  questionnaire  be  designed? 

Second,  is  information  needed  about  a  general 
phenomenon  (e.g.,  organization  climate,  job 
satisfaction)  or  specific,  clearly  defined  issues 
(e.g.,  what  kind  of  training  will  help  operators 
work  faster;  how  do  supervisors  get  training  on 
new  product  specifications)?  Third,  to  what 
extent  should  open-ended  and  closed-ended 
questions  be  used  C9:1003? 

Standardized  questionnaires  are  normally  used  when  the 
information  needed  by  the  interviewer  is  general  in  nature. 
Conversely,  original  questionnaires  are  normally  designed 
and  used  when  the  interviewer  needs  specific  information 
(9:100) . 

Questions  in  the  questionnaire  can  be  closed-ended, 
open-ended,  or  a  combination  of  both.  Closed-ended 
questions  result  in  responses  such  as  ”yes”  and  ”no”,  or 
"like”  and  "dislike”.  These  types  of  responses  are  easily 
tabulated  and  analyzed.  Open-ended  questions  ask  the 
respondent  to  provide  original  responses.  These  types  of 
responses  are  more  complicated  to  analyze  (9:100). 


The  choice  of  what  type  of  question  should  be  used 
depends  on  what  type  of  information  the  interviewer  is 
looking  for.  Maher  (9:1051  recommends  that  a  combination  of 
open-ended  and  closed-ended  questions  be  used  in  a 
questionnaire  so  as  to  get  a  clearer  response  from  the 
respondent  (9:1051. 


In  constructing  the  questionnaire,  the  researcher  is 
forced  to  determine  exactly  what  information  he  needs  to 
collect  and  how  he  is  going  to  collect  it.  Maher 
(9:105-1101  discusses  eight  steps  in  constructing 
questionnaires . 

The  first  step  in  constructing  questionnaires  is  for 
the  researcher  to  determine  what  information  he  needs  to 
know  so  that  he  can  ask  appropriate  questions  (9:1051. 

The  next  step  is  for  the  researcher  to  choose  the  type 
of  response  format  he  wants  to  use.  The  format  for  the 
questionnaire,  again,  can  include  either  open-ended 
questions,  closed-end  questions,  or  a  combination  of  both 
(9: 103-1041 . 

The  third  step  in  constructing  a  questionnaire  is  to 
determine  the  frame  of  reference  of  the  respondents.  During 
this  step,  the  researcher  must  determine  the  suitable 
vocabulary  for  the  respondent.  The  researcher  must  consider 
the  background  of  the  respondent,  such  as  how  well  informed 
the  respondent  is  and  whether  or  not  the  respondent  might  be 


biased  about  the  topic  (5:105). 

The  following  step  is  to  write  the  questions.  The 
questions  need  to  be  written  so  that  they  are  easily 
understood,  easy  to  answer,  and  ask  for  all  pertinent 
information  (3:106). 

The  fifth  step  is  to  prepare  a  summary  shBet .  A 
summary  sheet  is  a  tool  used  to  determine  how  the  researcher 
will  analyze  the  data.  A  summary  sheet  for  a  closed-ended 
question  survey  would  be  a  sheet  with  all  the  questions 
listed  on  it  and  adequate  space  alongside  each  question  to 
tally  the  number  and  types  of  responses  to  the  question 
(9:107) . 

Pretesting  the  questionnaire  on  other  people  and 

revising  it  if  necessary  is  the  next  step.  Pretesting  will 

help  the  researcher  to  determine  whether  or  not  thB 

questionnaire  asks  for  the  necessary  information  (9:10B). 

Pretesting  an  instrument  is  necessary  because,  as 
Backstrom  and  Hursch  (1563)  have  pointed  out,  "No 
amount  of  intellectual  exercise  can  substitute 
for  testing  an  instrument  designed  to  communicate 
with  ordinary  people'’  CB.-269D. 

The  seventh  step  is  to  construct  the  total 

questionnaire.  This  includes  thB  questions,  the  sequencing 

of  the  questions,  directions  on  how  to  fill  out  the 

questionnaire,  and  format  of  the  questionnaire  (9:10B). 

The  final  step  is  to  distribute  the  questionnaires.  In 

this  step  the  researcher  determines  the  amount  of  the 

population  to  be  sampled  and  submits  the  questionnaire  to 


them  (9:110). 


The  Nonresponse  Problem 


As  previously  noted  in  the  discussion  of  disadvantages 
of  questionnaires,  there  are  some  problems  associated  with 
using  questionnaires.  A  very  significant  problem  is  the 
error  induced  by  nonresponse.  ’’This  error  occurs  when  you 
are  not  able  to  find  those  whom  you  are  supposed  to  study” 
(4:1651.  Nonrespondents  include  people  who  either  don’t 
answer  the  questionnaire  or  answer  the  questionnaire 
incorrectly.  This  type  of  error  can  result  in  slanted  or 
meaningless  results. 

Yu  and  Cooper  C 15: 361  present  three  possible  solutions 
to  the  problem  of  nonresponse.  These  include  substituting 
someone  else  for  the  missing  respondents,  attempting  to 
determine  the  impact  on  the  survey  results  due  to 
nonrespondents  and  improving  the  research  design  tD  reduce 
the  number  of  nonrespondents . 

Yu  and  Cooper  recommend  that  the  researcher  improve  the 
research  design.  They  feel  it  is  the  preferable  solution 
because  ”it  is  an  attempt  to  eliminate  nonresponse  bias 
entirely  and  thus  avoid  the  untestable  assumptions  present 
in  other  solutions”  (15:361.  Their  recommendation  is 
supported  by  Erdos  (5:1431.  Erdos  says  ’’The  one  rule  which 
is  true  for  all  survey  methods  is  that  if  the  job  is  well 
planned  and  well  executed,  the  percentage  of  nonresponse 
will  be  small”  (5:1431. 


Summary 

A  questionnaire  is  a  survey  instrument  used  to  collect 
data.  The  use  of  questionnaires  has  its  advantages  and 
disadvantages.  The  design  of  the  questionnaire  has  a  large 
impact  on  the  usefulness  and  the  value  of  the  information 
obtained.  In  constructing  the  questionnaire,  the  researcher 
is  forced  to  determine  exactly  what  information  he  needs  to 
know  and  how  he  is  going  to  collect  it.  Finally,  a  very 
significant  problem  with  the  use  of  questionnaires  is  the 
error  induced  by  nonresponse . 


Appendix  C:  Appro ved  Survey 


DEPARTMENT  OF  THE  AIR  FORCE 

AIR  UNIVERSITY 

AIR  FORCE  INSTITUTE  OF  TECHNOLOGY 
WRIGHT -PATTERSON  AIR  FORCE  BASE  OH  4S433-6SS3 


«*yto  -  c  18  APR  1336 

ATTN  Of 

sopject  U.S.A.F.  Contract  Fro grammer  Information  Survey 
(CSAF  Survey  Control  Kumber  86-18) 


to  Survey  Participant 


1.  Often  personnel  newly  assigned  to  contract  pro  granting 
are  left  to  "learn  by  doing"  the  job  cf  contract  programing. 
They  stumble  through  by  the  process  cf  trial  and  error.  They 
are  told  to  read  regulations  and  ask  questions  of  other 
contract  programmers.  This  process  of  learning  can  be  very 
slow  and  frustrating  for  the  new  contract  programmer. 

2.  The  attached  survey  is  a  part  of  a  research  project  being 
done  to  determine  what  information  the  current  Air  Force 
contract  programmers  in  the  field  perceive  should  be  included 
in  a  handbook  designed  to  help  new  personnel  learn  the  job 

of  contract  programing  as  quickly  as  possible.  The  intent 
of  this  handbook  is  not  to  be  a  "how  to"  book  but  a  general 
information  book  for  new  personnel.  The  information  you 
provide  will  be  used  to  develop  an  outline  of  what  information 
should  be  in  the  book. 

3.  l’our  participation  is  voluntary.  Your  individual 
responses  will  be  combined  with  others  and  will  not  be 
attributed  to  you  personally. 

4.  Please  complete  the  attached  survey  and  return  it  tc 
AFIT/LSG  ir.  the  enclosed  envelope  within  five  working  days. 

If  you  have  any  questions,  contact  Captain  Donald  Kellogg 
at  AUTOVOK  785-5435.  Thank  you  for  your  cooperation  and 


2  Atch 

1.  Questionnaire 
cs  2.  Return  Envelope 


Copy  av.A'.uM 
penui!  .faJly  ]o 
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u*  tally  l-yil'S- 


a 


U.S.  AIR  FORCE  CONTRACT  PROGRAMMER 
INFORMATION  SURUEY 


PART  I 

Please  circle  the  letter  to  indicate  the  appropriate  answer. 

1.  I  am  currently  assigned  to 

a.  MAJCOM  Level 

b.  Base  Level 

2.  I  have  been  a  contract  programmer  for 

a .  0-6  months 

b.  6-12  months 

c.  12-18  months 

d.  More  than  18  months 

3.  I  work  far  the  Air  Force  as  a 

a.  Civilian  member 

b.  Military  member 

PART  II 

This  part  of  the  survey  relates  to  your  perception  as  to 
whether  or  not  certain  topic  areas  should  be  included  in  a 
handbook  designed  to  help  new  personnel  lBarn  the  job  of 
contract  programming  quickly . 

Topic  areas  are  listed  at  the  left  in  alphabetical  order, 
each  with  a  corresponding  set  of  numbers .  The  numbers  are 
a  five  point  increasing  scale.  The  scale  is  used  to  rate 
the  answer  to  the  question: 

Based  on  your  personal  experience,  which  topics  do  you 
recommend  for  inclusion  in  a  handbook  designed  to  help 
new  personnel  learn  the  job  of  contract  programming  as 
quickly  as  possible? 

The  numbers  on  the  scale  correspond  to  the  following 
perceptions: 

Cl)  Strongly  do  not  recommend 

(2)  Do  not  recommend 

(3)  Don't  know/No  opinion 

(4)  Recommend 

CS)  Strongly  recommend 

Please  indicate  your  opinion  by  circling  the  appropriate 
response . 


SCALE: 


Cl)  Strongly  do  not  recommend 

C2)  Do  not  recommend 

C3)  Don’t  knoui/No  opinion 

(4)  Recommend 

(5)  Strongly  recommend 


C- )  — >  C  +  ) 


1 .  Air  Force  Regulations  12345 

A  listing  and  description  of  the  regulations 
and  manuals  applicable  to  contract 
programming . 

2.  Airfield,  Noise,  and  Explosive  Safety  Waivers  12345 

A  description  of  what  each  of  these  waivers 
is,  their  purpose,  and  why  the  contract 
programmer  must  know  about  them . 

3.  Approval  Authorities  12345 


A  description  of  what  approval  authorities 
are,  their  legal  aspects,  and  how  they  impact 
the  programming  process. 

4.  Base  flaster  Planning  1  2  3  4  5 

A  description  of  what  base  master  planning 
is,  its  makeup  and  function,  and  how  the 
contract  programmer  interfaces  with  it . 

5.  Civil  Engineering  Contract  Reporting  System  12345 
CCECORS) 


A  description  of  what  CECORS  is,  its  purpose, 
what  information  it  provides,  and  what  the 
contract  programmer  would  use  the  information 
for . 

6.  Construction  Cost  Estimating  12345 

A  description  of  the  various  methods  of  cost 
estimating  Ci.e.,  like  structure,  square  foot 
costing,  systems  costing,  unit  pricing)  and 
the  role  of  the  contract  programmer  in  the 
preparation  of  a  cost  estimate. 

7.  Economic  Analysis  12345 

A  description  of  what  an  economic  analysis 
is,  its  purpose,  and  the  role  of  the  contract 
programmer  in  the  preparation  of  one. 
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SCALE:  Cl)  Strongly  do  not  recommend 

(2)  Do  not  recommend 

(3)  Don’t  know/No  opinion 

(4)  Recommend 

C 5 )  Strongly  recommend 

C- )  — >  C+) 

B.  Environmental  Planning  1  B  3  4  5 

A  description  of  what  environmental  planning 
is,  its  purpose,  and  how  the  contract 
programmer  interfaces  with  it. 

9.  Facilities  Board  1  B  3  4  5 

A  description  of  what  the  facilities  board 
is,  its  makeup  and  function,  and  the  role  of 
the  contract  programmer  in  preparing  for 
one . 

10.  Planning,  Programming,  and  Budgeting  System  1  B  3  4  5 

A  description  of  what  the  planning, 
programming,  and  budgeting  system  at  the 
national  level  is,  its  makeup  and  function, 
and  how  thB  contract  programmer  interfaces 
with  it. 

11.  Programming  Avenues  1  B  3  4  5 

A  description  of  the  various  programming 
avenues  Ci.e.,  Military  Construction 
Program,  Operations  and  Maintenance  Program, 
etc . )  including  the  type  of  work  they 
support  and  their  limitations. 

IB.  Project  Books  1  B  3  4  5 

A  description  of  what  project  books  are, 
their  purpose,  and  the  role  of  the  contract 
programmer  in  the  preparation  of  them. 

13.  Project  Identification  1  B  3  4  5 

A  description  of  how  projects  get  identified 
and  how  they  are  processed  before  they  reach 
the  contract  programmer . 

14.  Real  Estate  1  B  3  4  5 

A  description  t'f  the  real  estate  function, 

Its  purpose,  and  how  the  contract  programmer 
interfaces  with  it. 


7B 


PARI  III 


Appendix  D:  Difference  of  Sample  Means  Test 

The  difference  of  sample  means  test  C7:171)  is  used  to 
test  the  validity  of  the  null  hypothesis  CHo:  the  difference 
between  the  sample  means  of  two  different  populations  is  not 
statistically  significant).  ThB  alternate  hypothesis  CHa) 
in  this  test  is  that  the  difference  is  significant.  Sample 
means  C(1)  and  variances  CU5  are  calculated  for  each  of  the 
two  populations  CPI  and  P2)  and  then  are  incorporated  into 

f  nIIn,|'’  nr1  om  i  nnc  • 

w«.w  «-*-’•*  —  w— — .  .a  --n- - -  —  —  . 

a  -  WKP13  +  UCPg) 
m  n 

b  -  Ca)expl/2 

Z  -  UiZl)  _  fl \<J?£2 
b 

where 

UCP1)  -  variance  of  responses  in  PI 
UCP2)  “  variance  of  responses  in  P2 

m  -  total  number  of  responses  in  PI 

n  -  total  number  of  responses  in  P2 

flCPl)  -  arithmetic  mean  of  responses  in  PI 

MCP2)  ■  arithmetic  mean  of  responses  in  P2 

Z  “  test  value 

If  the  absolute  value  of  the  test  value  (Z)  is  greater  than 
the  critical  value,  as  determined  by  the  significance  level 
chosen,  then  the  difference  is  deemed  significant  or,  in 
other  words,  the  null  hypothesis  is  deemed  invalid. 
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An  example  of  this  test  procedure  is  as  follows: 

Ho:  Difference  of  sample  means  between  flAJCOn  personnel 
CPU  and  base  IbvbI  personnel  CP2)  for  the  topic  area 
of  Air  Force  Regulations  is  not  significant. 

Ha:  Difference  is  significant, 
where 

UCP1)  -  .4061 
UCP2)  -  .5552 
m  -  41 
n  “  152 
hCPlD  -  4.4B0 
nCP21  -  4.533 

therefore 

Z  -  -.33 

The  significance  level  chosen  for  this  test  was  .05,  as 
determined  bg  the  researcher’s  Judgment,  which  has  a 
critical  value  of  1.56.  Since  thB  absolute  value  of  Z  is 
less  than  the  critical  value,  the  conclusion  is  that  there 
is  no  significant  difference  between  the  means. 


Appendix  £:  Other  Suggestions  and  Comments 


The  following  is  a  consolidated  list  of  suggestions  and 
comments  provided  by  the  survey  respondents  with  regard  to 
specific  information  to  include  under  the  topic  areas 
identified  in  the  questionnaire  and  thB  manner  in  uihich  the 
handbook  should  be  organized.  Corrections  have  been  made 
for  grammatical  and  spelling  errors  only. 


Be  sure  item  11  includes  Red  Horse,  IMA  Teams  and  service 
contracts . 

Keep  book  in  a  loose  leaf  binder  to  allow  revisions  to  be 
accomplished  page  by  page. 

Under  Programming  Avenues,  item  11,  recommend  inclusion  of 
NAF,  private  sector  financing  CPSF),  Productivity  Investment 
Funds  CPIF5  and  other  fund  sources  where  competition  is 
based  on  economic  payback. 

A  brief  discussion  on  the  importance  of  keeping  channels 
open  with  the  in-house  folks  Ci.e.,  don’t  contract  the  Air 
Force  craftsmen  out  of  a  Job). 

Stress  the  importance  of  keeping  the  user  involved.  The 
programmer  is  there  to  serve  the  user . 

Stress  internal  coordination  among  programs,  engineering, 
and  customer  service.  UJe  frequently  shoot  our  own  foot 
without  any  help  from  the  users. 

Note  on  #1S.  At  some  bases,  contract  programmer  does  PB. 

At  some  bases,  design  engineer  does.  Some  bases,  it  is  a 
joint  effort. 

Do  not  agree  with  your  ’’general”  approach.  Yes,  be  general 
with  regard  to  related  areas  such  as  community  planning. 

But,  consider  that  you  could  provide  specific  information  on 
preparing  these  documents  that  is  not  to  be  found  elsewhere 
or  is  not  conveniently  explained  or  easily  understood. 

Need  to  emphasize  AFR  06-14  and  AFM  06-0.  Explain  what  an 
aircraft  parking  plan  is  and  apron,  taxiway,  and  runway 
criteria  also. 


Uery  feu  people  ere  Just  contract  programmers  and  in  fact  it 
is  not  desirable,  A  handbook  for  base  level  DEE  people 
uould  be  most  helpful  and  has  been  prepared  at  some  bases 
already.  This  narrou  effort  uould  seem  to  have  little 
merit . 

Suggest  use  of  matrix  type  of  format  to  delimit 
requirements,  references,  etc.  as  method  of  summarizing 
intricacies  of  programming. 

The  handbook  needs  to  be  designed  for  a  person  that  has  to 
do  programming  as  an  extra  duty  uhen  the  contract  programmer 
is  at  school  or  on  leave. 

ft  neu  programmer  uill  have  to  begin  programming  right  auay, 
so  include  examples,  regs,  etc.  so  he  can  begin. 

A  good  handbook  uill  become  a  resource  document  and  uill  be 
constantly  referred  to.  A  bad  handbook  uill  uaste  space  on 
a  shelf.  Be  careful  not  to  be  too  general  and  uhy  limit 
your  aim  to  the  Uneun  programmer?  In  E  months  he  uill  be  a 
veteran  and  uill  need  reference  material  rather  than  an 
introductory  handbook.  Ever  consider  a  "skeleton”  handbook 
in  looseleaf  format  that  could  be  personalized  and  added  to 
by  each  programmer?  It  might  be  uorth  considering. 

The  handbook  uould  have  to  be  updated  periodically  to  be 
useful . 

SAF/fll I  decisions  dealing  uith  uork  classifications. 

Each  base/MAJCQM/Air  Staff  uould  need  to  include  uork 
request/project  flou. 

Include  normal  flou  time  for  typical  riCP  and  minor 
construction  projects. 

Cover  major  agencies  involved  in  Non-Appropriated  Fund 
allocation  CAFWB,  local  MUR,  AAFES,  AFCOMS ,  Open  Messes, 
Private,  etc.). 

Recap  of  congressional  positions  ovBr  the  past  E-3  years  on 
particular  facilities  Ci.e.,  child  care  center). 

Private  sector  funding  potential 
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